NICE DSU TECHNICAL SUPPORT DOCUMENT 10: THE USE OF MAPPING METHODS TO ESTIMATE HEALTH STATE UTILITY VALUES

Size: px
Start display at page:

Download "NICE DSU TECHNICAL SUPPORT DOCUMENT 10: THE USE OF MAPPING METHODS TO ESTIMATE HEALTH STATE UTILITY VALUES"

Transcription

1 NICE DSU TECHNICAL SUPPORT DOCUMENT 10: THE USE OF MAPPING METHODS TO ESTIMATE HEALTH STATE UTILITY VALUES REPORT BY THE DECISION SUPPORT UNIT April 2011 Louise Longworth, 1 Donna Rowen 2 1. Health Economics Research Group (HERG), Brunel University, Uxbridge, Middlesex, UK. UB8 3PH. 2. School of Health and Related Research (ScHARR), University of Sheffield, Sheffield. Sheffield, UK. S1 4DA Decision Support Unit, ScHARR, University of Sheffield, Regent Court, 30 Regent Street Sheffield, S1 4DA Tel (+44) (0)

2 ABOUT THE DECISION SUPPORT UNIT The Decision Support Unit (DSU) is a collaboration between the Universities of Sheffield, York and Leicester. We also have members at the University of Bristol, London School of Hygiene and Tropical Medicine and Brunel University. The DSU is commissioned by The National Institute for Health and Clinical Excellence (NICE) to provide a research and training resource to support the Institute's Technology Appraisal Programme. Please see our website for further information ABOUT THE TECHNICAL SUPPORT DOCUMENT SERIES The NICE Guide to the Methods of Technology Appraisal i is a regularly updated document that provides an overview of the key principles and methods of health technology assessment and appraisal for use in NICE appraisals. The Methods Guide does not provide detailed advice on how to implement and apply the methods it describes. This DSU series of Technical Support Documents (TSDs) is intended to complement the Methods Guide by providing detailed information on how to implement specific methods. The TSDs provide a review of the current state of the art in each topic area, and make clear recommendations on the implementation of methods and reporting standards where it is appropriate to do so. They aim to provide assistance to all those involved in submitting or critiquing evidence as part of NICE Technology Appraisals, whether manufacturers, assessment groups or any other stakeholder type. We recognise that there are areas of uncertainty, controversy and rapid development. It is our intention that such areas are indicated in the TSDs. All TSDs are extensively peer reviewed prior to publication (the names of peer reviewers appear in the acknowledgements for each document). Nevertheless, the responsibility for each TSD lies with the authors and we welcome any constructive feedback on the content or suggestions for further guides. Please be aware that whilst the DSU is funded by NICE, these documents do not constitute formal NICE guidance or policy. Dr Allan Wailoo Director of DSU and TSD series editor. i National Institute for Health and Clinical Excellence. Guide to the methods of technology appraisal, 2008 (updated June 2008), London. 2

3 Acknowledgements The authors are grateful to John Brazier, Ben van Hout, Andrea Manca, Simon Pickard, Allan Wailoo and the team at NICE, led by Jennifer Priaulx, for their very helpful comments on an earlier draft. The editor for the TSD series is Allan Wailoo. The production of this document was funded by the National Institute for Health and Clinical Excellence (NICE) through its Decision Support Unit. The views, and any errors or omissions, expressed in this document are of the authors only. NICE may take account of part or all of this document if it considers it appropriate, but it is not bound to do so. This report should be referenced as follows: Longworth, L., Rowen, D. NICE DSU Technical Support Document 10: The use of mapping methods to estimate health state utility values Available from 3

4 EXECUTIVE SUMMARY The NICE Guide to the Methods of Technology Appraisals recommends that quality adjusted life years (QALYs) are used as the measure of outcome for economic evaluation, and that the EQ-5D is the preferred measure of health-related utility to calculate QALYs. The guide also recognises that EQ-5D data may not always be available to analysts producing submissions and reports for NICE. Where such data are not available, the guide states that mapping can be used to predict EQ-5D data. Mapping is the development and use of an algorithm (or algorithms) to predict health-state utility values using data on other indicators or measures of health. The algorithm can be applied to data from clinical trials, other studies or economic models containing the source predictive measure(s) to predict utility values even though the target preference-based measure was not included in the original source study of effectiveness. The predicted utility values can then be analysed using standard methods for trial-based analyses or summarised for each health state within an economic model. Although the use of mapping to predict utility data has only recently been referred to within the NICE Guide to the Methods of Technology Appraisals, it has been used for several years in NICE submissions. An overview of the use of mapping in Technology Appraisals recently found that mapping had been used in over a quarter of submission to the TA programme. This support document draws on previous research to report the methods that can be used to map to EQ-5D data and draws on existing research, conducted for the NICE TA programme and the wider literature. A series of recommendations are provided for analysts considering the use of mapping to estimate health-related utility for inclusion in NICE Technology Appraisals: In most cases, mapping should be considered at best a second-best solution to directly collected EQ-5D values, as the use of mapping will lead to increased uncertainty and error around the estimates of health-related utility. Mapping should be based on direct statistical association mapping rather than opinion. This involves two stages: firstly using empirical data regression models are used to estimate the relationship between target EQ-5D utility scores and other source indicators or measures of health. These models can be estimated specifically for the 4

5 TA or obtained from the mapping literature. Secondly the results of these models can be applied to patient level or mean level data from the clinical trial(s) or observation study containing the source predictive measure(s) in order to predict EQ-5D utility values. The characteristics of the estimation sample should be similar to the target sample for the mapping analysis, and should contain all variables from the target sample or included in the economic model that are thought to impact on EQ-5D scores. Under some circumstances, it may be appropriate for the estimation sample to include a broader range of people, providing that the target sample is sufficiently represented. Standard statistical techniques should be used to examine the data prior to mapping estimation to inform model selection and specification (for example frequency tables and correlations). The dataset used to estimate the mapping regression should be fully described including both the range of EQ-5D values and graphical plots showing the distribution of EQ-5D data. The range of observed EQ-5D values from the source sample and predicted EQ-5D values used in the cost-effectiveness model should be fully described to provide information of whether the EQ-5D predicted utilities have involved extrapolation, which should be avoided. The appropriate model type differs depending on the dataset and how it is applied. Standard econometric and statistical techniques and judgement based on prior knowledge of the clinical relationship between variables should be used to inform model selection and application (such as statistical significance, sign and size of coefficients, R-squared and adjusted R-squared, and information criterion of AIC and BIC). The properties of the sample dataset should be used to inform model selection and a justification should be provided explaining why the selected regression model was chosen. The statistical properties of the mapping algorithms should be clearly described. The root mean squared error or mean squared error should be reported. Errors should also be reported across subsets of the EQ-5D range (e.g. EQ-5D<0, 0 EQ-5D<0.25, 0.25 EQ-5D<0.5, 0.5 EQ-5D<0.75, 0.75 EQ-5D 1) and a plot of observed and predicted values should be used. 5

6 The model should be validated. Ideally this would be conducted using an external sample similar to the target sample. However it is unlikely that this will be available in many cases. Where the sample size is large enough to do so, it is recommended that the sample is randomly split to provide an estimation subsample and validation sample. The final model specification can then be re-estimated using the full sample. If there is no overlap in content between the measures of interest, mapping is unlikely to be able to appropriately capture the relationship to estimate health-related utility. Alternative methods for estimating health-related utility data should be considered in these circumstances. The main advantage of mapping is that it enables outcomes data collected in a trial or observational study to be used in economic evaluation to meet the NICE reference case, even if the main source trial or study did not include the EQ-5D. However, in most cases it is still preferable to obtain directly collected data within studies of effectiveness. Whereas most NICE submissions that have used mapping have used condition-specific measures of health related quality of life or clinical indicators of disease severity to predict EQ-5D scores for patients, the majority of published literature has focused on mapping between generic measures using general population samples. Whilst there are areas of uncertainty in the most appropriate methods for mapping, there is a significant amount of active research currently being conducted in this area and we hope that these will be resolved in the near future. 6

7 CONTENTS 1. INTRODUCTION AND BACKGROUND PURPOSE OF THIS TECHNICAL SUPPORT DOCUMENT THE NICE METHODS GUIDE WHAT IS MAPPING? NICE RECOMMENDATIONS ON MAPPING USE OF MAPPING IN NICE TECHNOLOGY APPRAISALS HOW TO USE MAPPING IN HEALTH TECHNOLOGY ASSESSMENT THE ESTIMATION SAMPLE THE MAPPING FUNCTION Model specification Model type Performance Uncertainty Use of mapping algorithms from the literature CONCLUSIONS SUMMARY OF RECOMMENDATIONS REFERENCES TABLES Table 1: Overview of mapping functions

8 INTRODUCTION AND BACKGROUND 1.1. PURPOSE OF THIS TECHNICAL SUPPORT DOCUMENT The Guide to the Methods of Technology Assessment (Methods Guide) describes key aspects of analyses submitted to the technology appraisals programme. 1 This Technical Support Document (TSD) is part of a wider initiative to produce a series of TSDs that accompany the Methods Guide. Each TSD describes how to use analytical techniques recommended in the Methods Guide, offers suggestions for analyses for areas not currently covered in the Methods Guide and identifies areas that would benefit from further methodological research. This TSD is concerned with the Measurement and valuation of health: the use of mapping to predict health-related utility data. Whilst the TSD looks at mapping health utilities in general, particular emphasis is placed on mapping to generate utility estimates for use in decision models in health technology assessment submissions to NICE THE NICE METHODS GUIDE The National Institute for Health and Clinical Excellence (NICE) Methods Guide provides guidance to researchers and analysts on NICE s preferred methods for conducting health technology assessments (HTAs) for its Technology Appraisals Programme. 11 NICE s preferred framework for economic evaluation is cost-utility analysis with quality adjusted life years (QALYs) as the main measure of health outcomes. The latest version of the NICE Methods Guide was updated in 2008 and provided more detailed guidance on the collection and use of health-related utility data for NICE HTAs than the previous methods guide. The 2008 Methods Guide states a preference for health-related utility data to be collected directly from patients using the EQ-5D, a generic and preference-based health-related quality of life (HRQL) measure. However, the Guide also recognises that such data may not always be available to analysts formulating health technology assessment (HTA) submissions to NICE. In these cases, the Guide recommends that consideration is given to mapping or cross-walking from other HRQL measures to the EQ-5D. Data using the EQ-5D instrument may not always be available. When EQ-5D data are not available, methods can be used to estimate EQ-5D utility data by mapping (also known as cross-walking ) EQ-5D data from other HRQL measures included in the relevant clinical trials(s). This can be done if an 8

9 adequate mapping function can be demonstrated and validated. Mapping should be based on empirical data and the statistical properties of the mapping function should be clearly described. (NICE Guide to the methods of technology appraisal, Section 5.4.6) 1.3. WHAT IS MAPPING? Mapping is the development and use of a model or algorithm to predict utility values using data on other indicators or measures of health. The data used to predict the health-related utility values could consist of condition-specific quality of life measures (such as the asthma quality of life questionnaire 2 ), other generic quality of life questionnaires (such as the SF-36 3 ), clinical indicators of disease severity (such as the Canadian Cardiovascular Society Score for angina 4 or the Psoriasis Area and Severity Index 5, socio-demographic variables or a combination of these. Data on the target preference-based measure (e.g. EQ-5D) and the source predictive measure(s) (the indicators or measures of health that will be used to map to the preference based measures) must be collected within a sample. From these sample data, models can be developed to estimate the relationship between the target measure and the other indicators or measures of health. Where the target measure is a multi-attribute classification system such as EQ-5D, the data can be mapped to either the index utility value or the individual dimension responses. The models can then be applied to data from the clinical trial or other studies containing the source predictive measure(s) to predict utility values even though the target preference-based measure was not included in the original study. The predicted utility values can then be analysed using standard methods for trial-based analyses or summarised for each health state within an economic model. In its simplest form, mapping can be considered equivalent to taking the mean value for a given health state. For example, consider the case of a condition categorised into two health states: stable disease and progressive disease. If EQ-5D data and the health state category were collected for a sample of patients, we could estimate the mean EQ-5D value for patients at each of the two stages of disease. These mean values could then be assigned to patients in a trial in which the stage of disease is recorded. However, simply using a mean value (and distribution where reported) for a similar broadly-defined health state from another dataset or the reported literature can mask variation between patients. 9

10 Whether the mapping approach will offer an advantage over simply using mean values from an external dataset will, in part, depend on the structure of the economic model being used to reflect the decision problem. If the model has a simple structure with few health states, then reliable estimates of the mean (and variance) of the EQ-5D values associated with those health states may suffice. However where there are multiple predictors of health status that can be measured and reflected in the decision-model, then the mapping approach can predict the health-related utility value more accurately. For example, if the health states in a model are defined according to a 20-point measure of disease severity, it may not be possible to obtain EQ-5D values for each of those 20 levels of severity from a sufficient number of patients. However providing there is a predictable relationship between the EQ-5D and the severity measure the relationship between the measures can be estimated based on all the data in order to provide EQ-5D estimates for each of the 20 health states. Mapping also enables the health-related utility data to be linked directly back to data collected within the clinical trial(s) used to inform the estimates of cost effectiveness. Mapping is most commonly used in NICE submissions where utility data have not been directly collected within the clinical trials of the treatments under consideration. However, mapping techniques can also be used to incorporate utility data collected directly within the main clinical trial of interest into economic models, where the structure of the model is driven by other outcome measures. An economic model may have been constructed to define health states using a clinical measure of disease severity. In this case, mapping techniques can be used to explain the relationship between the two measures and to estimate the utility value (or distribution of values) associated with a health state defined by the clinical measure. An alternative approach would be to simply estimate the mean and variance for each of the health states described by the model from the data collected. For example, in the case of treatment for rheumatoid arthritis, the Health Assessment Questionnaire (HAQ) 6 is a commonly used measure of clinical outcomes. Several studies have sought to explain the relationship between health-related utility and HAQ scores using mapping type methods (see 7 for a recent overview). It is possible to use this approach even when utility data have been collected directly within the primary source/s for clinical effectiveness, as a means of incorporating the data within the economic model. However, concerns have been expressed when EQ-5D data have been used in this manner in one NICE Technology Appraisal. In this appraisal concerns were expressed by the independent reviewers and Advisory Committee 10

11 that the estimated mapping function did not accurately reflect the observed data from the trial when applied to the model (NICE TA198). This guide focuses solely upon mapping using direct statistical association, and mapping using expert opinion to convert between measures is not recommended for submissions to NICE technology Appraisals NICE RECOMMENDATIONS ON MAPPING There are various options available to the analyst when considering the use of mapping for the estimation of health state utilities for HTA. What measures to map to and which measures to map from? What form should the model underpinning the algorithm take? Which statistical methods should be employed? How should results be tested, validated and reported? In terms of which measure should be used as the outcome of the mapping exercise, the NICE Methods Guide 2008 states a preference for mapping to the EQ-5D (Guide to methods of technology appraisal, Section 5.4.6). 1 This preference, as with the preference for EQ-5D generally, is based on a need for consistency across NICE appraisals. The Methods Guide is not restrictive about the source measures from which the EQ-5D utility data are predicted. The Guide refers to measures included in the clinical trials, however this is not restrictive and other types of studies may be more appropriate in some circumstances, for example if the events of interest are rarely observed within a trial setting. The Methods Guide also states that mapping should be based on empirical data. This means that both the EQ-5D and the measure/s used to map from, are administered in a sample of people to generate empirical data, rather than researchers (or others) attempting to map from an measure to the EQ-5D based only on judgement or the face value of the measures. The Guide also states that the adequacy of the mapping function should be demonstrated and validated, and that the statistical properties of the function should be described (Guide to methods of technology appraisal, Section 5.4.6). 1 However, it is not prescriptive in stating which statistical or other tests should be undertaken to demonstrate the adequacy and validity of the resulting algorithm, nor does it stipulate the conditions under which an algorithm has not been demonstrated as valid. 11

12 Section 2 summarises the use of mapping in HTAs for NICE to date. Section 3 of this document draws on the existing literature and experience with the use of mapping to discuss the various approaches that can be taken and to make recommendations for NICE HTAs based on the evidence to date. Section 4 summarises the recommendations. 2. USE OF MAPPING IN NICE TECHNOLOGY APPRAISALS The 2008 edition was the first of the NICE Methods Guides to suggest mapping as a potential solution for an absence of health state utility data. Previous Methods Guides cited a preference for self-assessment of health status by patients 8 and data from validated generic preference-based measures, 9 but did not offer guidance to the analyst on how to conduct costutility analyses if such data had not been collected within clinical studies. Although not mentioned in the earlier Methods Guides, the use of mapping for NICE HTAs is not new and several NICE submissions have previously included mapping as a method of estimating health-related utility values. Two reviews of the health-related utility data included in NICE Technology Appraisals have been published. 10,11 An early review of independent assessment reports produced for the Technology Appraisals Programme up to May 2003 by Stein and colleagues found that mapping approaches had been used in NICE appraisals. 12 The review covered 56 appraisals; 28 of which reported 45 cost-utility analyses. The authors report two clear cases of mapping in appraisals: one where data from the Health Assessment Questionnaire (HAQ) were mapped onto the EQ-5D in an appraisal of treatments for rheumatoid arthritis and; another whereby data from the Child Health Assessment Questionnaire (CHAQ) were mapped to the EQ-5D in an appraisal of etanercept for juvenile idiopathic arthritis. The methods used to undertake the mapping were reported to be limited or not reported at all. Stein and colleagues identified a further five appraisals where health states had been mapped to preference-based questionnaires using opinion rather than empirical data. In two cases the mapping was conducted on the basis of clinical opinion, in one case it was based on the opinions of the HTA analysts and no details were provided in the remaining two cases. All but one of the HTAs mapped health states onto the EQ-5D; the other HTA mapped health states to the Index of Health-Related Quality of Life (IHQL). 12

13 Another review of health-related utility data included in NICE submissions has recently been published. 11 This included 46 appraisals conducted from the time of the implementation of the 2004 Methods Guide up to the time that the current Methods Guide was introduced in Thirty-nine appraisals included cost-utility analyses; and when including both independent and sponsor submissions, this accounted for 71 individual cost-utility analyses submitted to NICE. The review found that the use of mapping had increased since the previous review to 27% (n=19) of submissions over the period of the review. Empirical data were used to generate the mapping mechanism in 16 submissions, one was based on expert opinion and the methods used in the other two submissions were unclear. Six of the submissions used published mapping algorithms in their analyses, and a further appraisal used an existing, but unpublished, algorithm. The majority of submissions included analyses that mapped to the EQ-5D (n=14). Other end-points for the mapping process included the Health Utilities Index (HUI) (n=2), the SF-6D (n=1) and patients time trade-off values for their own health (n=2). In most cases health state utility data were mapped from conditionspecific measures (n=14); the remainder mapped from generic HRQL measures (n=2), nonstandardised vignettes of health states (n=1) or the details were unclear (n=2). A further 44 Technology Appraisals have been published over the two year period since the publication of the updated NICE Methods Guide in June These recent appraisals have been reviewed to assess the use of mapping since the new Methods Guidance was issued (although it should be recognised that some of the analyses would have been conducted prior to the publication of the updated Guide). The same methods were used to identify and extract the data as reported in the previous review. 11 Of the 44 appraisals, four included HTAs which used mapping to estimate health-related utility data. All four of the appraisals based the mapping algorithm on empirical data. They were based on previously developed mapping algorithms that were publicly available as fully published studies (n=2), in abstract form (n=1) or from a previous NICE HTA report (n=1). All four HTAs mapped from a condition-specific measure of either HRQL or disease severity. Half of these analyses mapped data to EQ-5D utilities and the other half mapped to patients time trade-off values of their own health. The submissions contained little information about the statistical properties of the mapping algorithms; however they did provide references to the original documents which described how the algorithms were developed. 13

14 It is evident from the reviews of NICE Technology Appraisals that mapping was used to estimate utilities for NICE HTA submissions prior to being explicitly referred to in the 2008 Methods Guide. The reviews suggest that mapping has been used in submissions from the very early beginnings of NICE, but has become more common since the publication of the 2004 version of the NICE Methods Guide. An update of the earlier review conducted specifically for this paper, has found that fewer HTAs used mapping to estimate healthrelated utility data since being recognised as a potential second best solution in the 2008 Methods Guide. The early review of NICE appraisals found several instances of mapping based on the opinions of researchers or health care professionals. The later reviews show that this has become less common, with most recent mapping analyses being based on empirical data. However, the level of detail with which the mapping algorithms and analyses have been presented in the documentation has been generally poor, with few details of the statistical performance of the mapping algorithms being presented to the NICE Technology Appraisal Committee. 3. HOW TO USE MAPPING IN HEALTH TECHNOLOGY ASSESSMENT As stated previously, mapping enables data available in the trial(s) (non-preference-based HRQL measures, preference-based measures, clinical measures, socio-demographic data) to be used to estimate EQ-5D utility scores. Mapping involves three stages. Firstly a separate estimation dataset is required that contains the data that you are mapping from, the source, and the target preference-based measure. Secondly regression methods are used to map this data onto either the index score or the classification system of the target measure. Thirdly the regression results are applied to the trial(s) or observational study dataset to estimate utility scores for the target measure at either the mean or observational level. Ideally, a validation stage should also be applied, whereby the regression results are validated against another dataset. Each of these stages will be discussed in turn below with reference to published literature on mapping. This technical support document focuses upon mapping to EQ-5D, as this is stated as the preferred measure for NICE Technology Appraisals. However, the approaches described below could apply to other health-related utility measures. 14

15 3.1. THE ESTIMATION SAMPLE The generation of the mapping function involves the estimation of the statistical relationship between the target measure (the EQ-5D) and the predictive measure(s) using an estimation dataset. The first step in the mapping approach is to obtain the estimation sample. As this assumes that the statistical relationship is the same across the estimation and trial datasets, the choice of the estimation sample is crucially important. The estimation sample is the group of people, usually patients, who will complete the EQ-5D to report their own health and from whom data on the source measures will also be obtained. In order to be confident about the generalisability of the mapping function to the target sample, the clinical and demographic characteristics of people in the estimation sample should be as similar to the characteristics of the target sample to which the mapping algorithm will be applied as possible. All covariates used in the mapping function should be overlapping in distribution for the estimation and target samples. It is recommended that all variables included within the target source (e.g. the main clinical trial/s used to inform clinical effectiveness within the economic model) that are thought likely to impact on EQ-5D values should be included in the estimation sample. If no existing dataset is available that includes both the source and the EQ-5D it will be necessary to collect the data to estimate the mapping regression. One study that uses this approach followed the same inclusion and exclusion criteria for recruiting the estimation sample as the clinical trial which was the target for the mapping. 13 It may also be possible to use an estimation sample including a wider range of observations, providing that the full range of clinical and demographic characteristics are captured within the estimation sample (see for example 14,15 ). However, the ability of the mapping algorithm to predict the utility values of the target population will depend upon its relevance to that population. For example, there are several papers mapping the cancer-specific EORTC QLQ-C30 onto EQ-5D for different cancer patient groups. The mapping regressions differ between the different patient samples used to estimate the mapping algorithm, and one possible explanation is that the algorithms may not be generalisable to different cancer conditions This is an area that requires further research. Alternatively double mapping has been used in unpublished studies, where it has not been possible to obtain a dataset that contains both the source and target measures. This involves one estimation dataset to map from the source to an intermediary measure and a second 15

16 estimation dataset to map from the intermediary measure onto the target measure. These estimates should be treated with caution as the process generates intermediary measure estimates that are then used to generate estimates, meaning that mapping twice is likely to increase the error and uncertainty around the EQ-5D estimates. If this approach is taken, the uncertainty should be fully accounted for within the economic analysis THE MAPPING FUNCTION Model specification The model specification can take a number of forms depending on which best suits the data and the decision problem at hand. The independent variable could be the utility index value or the responses to the EQ-5D dimensions. The explanatory variables should be those which best predict the EQ-5D values for health states included in the economic analysis. Additive models are currently most commonly used, however alternative model specifications have been used in the literature. A recent review of mapping studies undertook a systematic search of the literature supplemented by unpublished studies (identified by contacting researchers) in early 2007 and reports on 30 studies covering 119 mapping models. 20 As with the review of NICE submissions (Section 2) the review of the literature found that the most common target measure was EQ-5D (15 studies). However, in contrast to the review of NICE submissions in which the source measures were largely condition-specific, the most commonly used source measures in the literature were SF-36 (7 studies) and SF-12 (6 studies),. The most common model specification involved the use of a preference-based index as the dependent variable and dimension or item scores as independent variables. Papers also examined model specifications including squared terms and interaction terms to explore possible non linear relationships between the target and source measures. The review found that these had little impact, but it is likely that this differs by source and target measures, patient group and patient severity. The review found that the inclusion of non-health variables such as sociodemographics made some improvement in the accuracy of the mapping function. Table 1 provides a summary of different model specifications for the mapping function. 16

17 Table 1: Overview of mapping functions Dependent variables Independent variables Model selection and specification Model type Performance Validation EQ-5D index EQ-5D dimension levels Condition specific measure: Overall score, summary scores, item level scores, item level dummies, interaction terms, squared terms, cubic terms Generic measures: Overall score, summary scores, item level scores, item level dummies, interaction terms, squared terms, cubic terms Clinical measures: overall score, summary score, categorical dummies Socio-demographic variables Other relevant health data Use prior knowledge of clinical relationships Use standard statistical techniques to examine the data prior to mapping estimation (e.g. frequency tables and correlations) Fully describe the dataset used to estimate the regression model including both range of EQ-5D and plots showing EQ-5D distribution Fully describe the range of EQ-5D predicted values used in the costeffectiveness model Linear ordinary least squares (OLS) Tobit Censored least absolute deviation (CLAD) Two part model (TPM) Generalized linear model (GLM) Latent class mixture model Censored mixture model Multinomial logit model Goodness of fit: Statistical significance, sign and size of coefficients R-squared and Adjusted R- squared Information criterion of AIC and BIC Further tests of model fit such as Ramsey RESET test, Park test, Jarque-Bera test Plots to examine whether model assumptions are valid Predictive ability: Root mean squared error (RMSE) and mean squared error (MSE) RMSE, MSE, mean error, mean absolute error by subset of severity range of EQ-5D and/or predictive measure(s) Application and assessment of mapping algorithm when applied to a validation sample Validation sample can be a separate patient sample to the estimation dataset or the dataset used to estimate the mapping function can be randomly separated into estimation and validation samples Plots of observed and predicted EQ-5D scores 17

18 The recent review of mapping studies found that explanatory power using R-squared was often low for models that involved mapping a condition-specific measure onto a generic preference-based measure and errors were often larger than for models mapping a generic measure onto a generic preference-based measure. 20 This may occur due to limited conceptual overlap as important dimensions in the condition-specific measure may not appear in the generic measure and vice versa. The estimation of the mapping regression relies upon statistical dependence between the EQ- 5D and the source measures, and the avoidance of omitted variables. If the source measures have little conceptual overlap with the dimensions of the EQ-5D the regression model may suffer from omitted variable bias, have poor explanatory power and large prediction errors. This can undermine the model and the uncertainty around the predicted values may be substantial. Where the EQ-5D is shown to not adequately capture the impact of the condition or treatment, it may be necessary to consider using an alternative approach to utility estimation (see TSD8 21 and TSD11 22 ). The selection of explanatory variables should be based on a combination of judgement based on prior knowledge of the clinical relationships between variables, and standard statistical and econometric techniques. Consideration should be given to the variables that are expected to impact on the health-related utility of people with the condition of interest. This can be based on patient and clinical opinion obtained directly or reported in the literature. Decision rules for the inclusion of variables should be specified a priori, such as levels of statistical significance and the signs of the coefficients matching prior stated beliefs. Correlation should be used to examine the relationship between source and target measures, and if there is poor correlation this indicates that the mapping function will perform poorly (see 23 for an example of this). Akaike s Information Criterion (AIC) 24 and the Bayesian Information Criterion (BIC) 25 can be used to inform the choice of model specification (see for example 7 ). Other tests should also be used to enable the researcher to define a robust model, such as examining the extent to which the model suffers from misspecification (for example Ramsey RESET test 26 ), omitted variables and heteroscedasticity (for example the Park test 27 ) or nonnormality in the errors (for example the Jarque-Bera test 28 see 29 for an explanation of its usage in panel data). The severity of the condition reflected in the source measure should also be captured by the target measure. If the source measure cannot capture the same health problems captured by 18

19 the mild, moderate and severe levels of the EQ-5D it will not be able to accurately predict these levels. The extent to which this is a problem will depend on the severity range of the target and source measures, and the severity range of the estimation and trial dataset. The dataset used to estimate the mapping regression should be fully described including both the range of EQ-5D values and graphical plots showing the distribution of EQ-5D data. The range of EQ-5D predicted values used in the cost-effectiveness model should also be fully described to provide information of whether the predicted EQ-5D utilities have involved extrapolation Model type The appropriate model type differs depending on the dataset and how it is applied. It also depends on whether the aim is to predict the EQ-5D index value or whether it is to predict responses to each of the five dimensions of health described by the EQ-5D. As specified in section the properties of the regression dataset should be clearly outlined. These properties should then be used to inform model selection and a justification provided explaining why the selected regression model was chosen. Mapping to EQ-5D index values The model type used to map source measures onto EQ-5D should take into account the distribution of EQ-5D utilities in the estimation dataset. The EQ-5D index has been shown to exhibit ceiling effects, meaning that typically EQ-5D datasets have a substantial proportion of people reporting full health with an EQ-5D value at 1. Although the distribution of EQ-5D index values varies by patient group and study, often a bimodal or trimodal distribution is observed, with one peak around full health, one peak for moderate states and a further peak for more severe states. The recent mapping review 20 found that the most common estimation technique was ordinary least squares (OLS), yet linear regressions may not always accurately predict the EQ-5D distribution for high and low EQ-5D values. 15,20 Some of the standard model specifications have been shown to predict fewer values towards the extremes of the utility scale, even where they are evident in the observed source data. OLS has been criticized in particular as being inappropriate for regressions mapping onto EQ-5D due to the bounded nature of the EQ-5D as by definition people cannot have an EQ- 5D utility value higher than 1, which represents full health. In addition, the standard UK value set has a lowest possible value of The OLS model does not restrict the range of values and therefore may lead to implausible predicted values outside of the existing range of EQ-5D values. Researchers have explored alternative types of models to overcome the 19

20 theoretical limitations of OLS models for the analysis of EQ-5D data, including tobit 15,30,31 and CLAD (censored least absolute deviation). 15,30-32 The results of this research has been mixed with some concluding that CLAD provides an improvement in model performance compared to OLS, 15,30,31 others stating that the improvement of CLAD over OLS is small, 18 and the review of mapping studies found that the use of tobit and CLAD had little impact. Most of the models are based on mean values, apart from CLAD which is a median model. The choice between the use of mean and median values requires normative judgements as well as statistical considerations. Health state valuation for economic evaluation for decisionmaking has been mainly based on mean models to date, however there has been some recent research utilizing median models. 19,33-35 The choice and application of alternative models is an area of recent and ongoing research and a large number of models have been recently explored in the mapping literature. This includes the use of models to address the EQ-5D ceiling effect including a generalized linear model, 36 a latent class model, 32 a two-part or two-step model (TPM), 32,36,37 and a random effects censored mixture model. 7 The first part of the two-part model uses a logit regression to estimate the probability that an individual (at the observational level) is in full health and the second part estimates EQ-5D utilities for individuals who are not in full health using either OLS, 32,36,37 a generalized linear model (GLM) 36 or a log-transformed EQ-5D index (TPM-L). 32 One paper addresses over-prediction for severe health states by estimating separate regressions for these states and using cut-off points on the source measure to identify which model should be used to predict EQ-5D at the observational level. 38 The results from this recent research have been mixed. The studies estimating these models found that the TPM and GLM models do not seem to offer an improvement on OLS in terms of performance. One study found that OLS had superior performance to both GLM and the two-part model. 36 Another study found that OLS regression was more accurate at estimating the group mean than the CLAD model, multinomial logit model and TPM, yet the accuracy deteriorated in older and less healthy subgroups and for these the TPM performed better. 37 The latent class model can handle data where there are more than two classes in the data, so is more flexible to deal with the tri-modal distribution of EQ-5D data. One study 32 found that the latent class model and TPM-L performed better than OLS, CLAD, and a TPM using OLS in the second stage. A adjusted censored mixture model has been used to deal with the bimodal or tri-modal EQ-5D distribution and although high errors were observed the authors concluded that the method offers a vast improvement in performance in comparison to OLS and tobit based on other selection criteria 7. Further research using the latent class model, 20

21 TPM-L and random effects censored mixture model is encouraged, especially for smaller patient datasets as existing research has been conducted on relatively large datasets which may not be typical for the datasets used to estimate mapping functions for NICE submissions. 7,32 Mapping to the EQ-5D dimension responses Although the health-related utility values produced by the EQ-5D value sets are usually treated as continuous, in practice they take a limited number of discrete values. An alternative approach is to map to the descriptive system of the measure, which enables the value set to be applied separately and therefore may better reflect the distribution of values that would have been obtained if collected directly. In addition, if analysts are conducting technology appraisals for agencies in addition to NICE, value sets from other countries can be applied to the predictions from the mapping exercise. The most commonly used approach to mapping to the EQ-5D dimensions has been through the use of logistic regression. Some papers reported using a multinomial logit model to estimate separate mapping functions to predict the level of each dimension of the target preference-based measure, and then applying the standard published value sets to obtain utilities. 37,39 However papers comparing this approach to other approaches found that it did not offer an improvement. 15, Performance Measures of explanatory power such as R-squared report how well the mapping function explains the variation in EQ-5D utilities in the estimation dataset. Although this is a useful indicator of performance it does not show whether the mapping function is equally appropriate across the entire range of EQ-5D utilities. If the aim of mapping is to estimate EQ-5D utilities when EQ-5D data are unavailable from the primary source of effectiveness, the accuracy of predictions is a key aspect of performance. Mean absolute error (mean absolute difference between estimated and observed EQ-5D utilities) and root mean squared error both indicate the error in the estimates in the dataset used to estimate the regression and smaller errors are preferred. Whilst these errors are not necessarily representative of the errors in the estimates when the results are applied in the separate dataset, they can provide some indication of how large the errors are expected to be. Some mapping studies have reported under-prediction for very mild EQ-5D utilities and over-prediction for more severe states, 15,40 with better prediction for mild and moderate states. However in the literature surprisingly few studies report error across subset range, meaning that the true extent of the problem cannot be determined. Errors should be reported 21

22 across subsets of the EQ-5D range (e.g. EQ-5D<0, 0 EQ-5D<0.25, 0.25 EQ-5D<0.5, 0.5 EQ-5D<0.75, 0.75 EQ-5D 1) and a plot of observed and predicted values should be used. These are useful for indicating whether there is systematic bias in the predictions and whether heteroskedasticity is present. If there is systematic bias in the predictions, consideration will need to be given to how it impacts on the results of the cost-effectiveness analysis. For example, whether it is likely to be more significant for specific subgroups of patients or for one intervention compared to another. In addition errors reported across subsets of the range of the predictive measure(s) (see 7 for example) can inform application of the mapping algorithm in the trial dataset and reporting this should be considered. Some studies divide their estimation dataset into two samples; an estimation sample and a validation sample (for example ). The mapping function is estimated on the estimation sample and its performance is examined using the validation sample. This has the advantage that it assesses the mapping function by its prime purpose, however it reduces the sample size of the estimation sample. A randomly allocated split of the data should enable the analyst to assess how well the algorithm predicts the health state values for the validation sample. If predictive ability is poor when assessed based on a non-random split of the data, it may not be possible for the analyst to judge whether the poor performance is due to the functional form of the model or a lack of generalisability to a systematically different population. The reduced precision in the coefficients of the mapping function from the reduction in sample size may be overcome by re-estimating the mapping model using the full dataset once the specification of the model has been assessed using the split-sample approach. Furthermore if the division of the estimation dataset into two is truly random the model is expected to perform well, yet this does not necessarily indicate that it will perform similarly when applied to the trial data if the characteristics of the sample are different to the estimation and validation samples Uncertainty The uncertainty in health-related utility values should be incorporated into economic analyses as for all other parameters. There are different sources of uncertainty in the values estimated from mapping analyses. The parameter uncertainty in the estimated regression analysis should be taken into account using the standard error and correlation in probabilistic sensitivity analysis. However, where mapping has been used to predict values for a sample of patients where the data have not been directly collected there is also uncertainty in the mapped values because they are predicted rather than directly reported. Furthermore, some 22

23 researchers have shown that the confidence intervals around the predicted values as a result of mapping tend to be narrower than confidence intervals around directly observed values. 44,45 If there are multiple possible mapping functions, these can be applied in sensitivity analyses to give an indication of the uncertainty associated with the choice of algorithm. However alternative algorithms capturing plausible forms of the relationship between the predicted utility values and alternative explanatory variables may not be available. Further research is needed to establish the best ways of capturing all of the uncertainty in the mapped utility estimates. Until then analysts and decision-makers should be aware that uncertainty around mean mapped estimates may be underestimated Use of mapping algorithms from the literature Generating a de novo mapping function gives the analyst control over the inclusion and exclusion criteria for their estimation sample, and therefore influence over the generalisability of their mapping function to their target population. However, existing mapping functions may be available in the literature to the analyst. In these circumstances we recommend that careful consideration is given to the generalisability of the mapping function to the target population, including the range of disease severity over which the function was estimated and the potential for systematic differences in the populations that could impact on the health state utility values. Most of the considerations outlined above would also apply to the review and use of published algorithms. There may be circumstances in which all the variables included in the published algorithm are not available to the analyst in their dataset. Applying these algorithms are still theoretically possible by applying mean values to these variables, however this reduces the granularity in the resulting estimates. 4. CONCLUSIONS The main advantage of mapping is that it enables outcomes data collected in a trial or observational study to be used in economic evaluation to meet the NICE reference case, even if the study did not include the EQ-5D. Preferably, EQ-5D data should be collected directly to reflect the impact of treatment on overall HRQL, rather than just on the variables used to estimate the mapping algorithm. For example, if the mapping algorithm includes only a clinical measure, the mapping function may not reflect the impact of other effects of treatment that are not captured by the clinical measure. In addition, uncertainty and errors around the estimates can affect the accuracy of the EQ-5D utilities when used in economic evaluation. 46 However, there may be exceptions where other data sources are most 23

NICE DSU TECHNICAL SUPPORT DOCUMENT 12: THE USE OF HEALTH STATE UTILITY VALUES IN DECISION MODELS

NICE DSU TECHNICAL SUPPORT DOCUMENT 12: THE USE OF HEALTH STATE UTILITY VALUES IN DECISION MODELS NICE DSU TECHNICAL SUPPORT DOCUMENT 12: THE USE OF HEALTH STATE UTILITY VALUES IN DECISION MODELS REPORT BY THE DECISION SUPPORT UNIT July 2011 Roberta Ara, Allan Wailoo School of Health and Related Research,

More information

PROPOSED REVISIONS TO THE PATIENTS ACCESS SCHEME. April 2012. Sarah Davis School for Health and Related Research, University of Sheffield

PROPOSED REVISIONS TO THE PATIENTS ACCESS SCHEME. April 2012. Sarah Davis School for Health and Related Research, University of Sheffield RANIBIZUMAB AND PEGAPTENIB FOR THE TREATMENT OF AGE- RELATED MACULAR DEGENERATION (TA 155): IMPACT OF PROPOSED REVISIONS TO THE PATIENTS ACCESS SCHEME REPORT BY THE DECISION SUPPORT UNIT April 2012 Sarah

More information

NICE DSU TECHNICAL SUPPORT DOCUMENT 11: ALTERNATIVES TO EQ-5D FOR GENERATING HEALTH STATE UTILITY VALUES

NICE DSU TECHNICAL SUPPORT DOCUMENT 11: ALTERNATIVES TO EQ-5D FOR GENERATING HEALTH STATE UTILITY VALUES NICE DSU TECHNICAL SUPPORT DOCUMENT 11: ALTERNATIVES TO EQ-5D FOR GENERATING HEALTH STATE UTILITY VALUES REPORT BY THE DECISION SUPPORT UNIT March 2011 John Brazier, Donna Rowen School of Health and Related

More information

August 2012. Authors: Ben Kearns, Roberta Ara, Allan Wailoo School of Health and Related Research, University of Sheffield

August 2012. Authors: Ben Kearns, Roberta Ara, Allan Wailoo School of Health and Related Research, University of Sheffield A REVIEW OF THE USE OF STATISTICAL REGRESSION MODELS TO INFORM COST EFFECTIVENESS ANALYSES WITHIN THE NICE TECHNOLOGY APPRAISALS PROGRAMME REPORT BY THE DECISION SUPPORT UNIT August 2012 Authors: Ben Kearns,

More information

Journal Article Reporting Standards (JARS)

Journal Article Reporting Standards (JARS) APPENDIX Journal Article Reporting Standards (JARS), Meta-Analysis Reporting Standards (MARS), and Flow of Participants Through Each Stage of an Experiment or Quasi-Experiment 245 Journal Article Reporting

More information

TOCILIZUMAB FOR THE TREATMENT OF RHEUMATOID ARTHRITIS

TOCILIZUMAB FOR THE TREATMENT OF RHEUMATOID ARTHRITIS TOCILIZUMAB FOR THE TREATMENT OF RHEUMATOID ARTHRITIS REPORT BY THE DECISION SUPPORT UNIT 2 nd September 2011 Jon Minton, Paul Tappenden, Jonathan Tosh School of Health and Related Research, University

More information

TAR call-off contract Scope of Work, Templates

TAR call-off contract Scope of Work, Templates NICE TAR Protocol template guide 1. Introduction 1.1 In order to support the National Institute for Health and Clinical Excellence s (NICE) preferred methods 1 a protocol template for Technology Assessment

More information

NATIONAL INSTITUTE FOR HEALTH AND CARE EXCELLENCE CENTRE FOR HEALTH TECHNOLOGY EVALUATION. Technology Appraisals Programme

NATIONAL INSTITUTE FOR HEALTH AND CARE EXCELLENCE CENTRE FOR HEALTH TECHNOLOGY EVALUATION. Technology Appraisals Programme Page 1 of 14 NATIONAL INSTITUTE FOR HEALTH AND CARE EXCELLENCE CENTRE FOR HEALTH TECHNOLOGY EVALUATION Technology Appraisals Programme Specification for Cancer Drugs Fund data collection arrangements 1

More information

Patient Reported Outcome Measures (PROMs) in England: Update to reporting and case-mix adjusting hip and knee procedure data.

Patient Reported Outcome Measures (PROMs) in England: Update to reporting and case-mix adjusting hip and knee procedure data. Patient Reported Outcome Measures (PROMs) in England: Update to reporting and case-mix adjusting hip and knee procedure data. Insert heading depending on Insert line heading length; please depending delete

More information

Credit Card Market Study Interim Report: Annex 4 Switching Analysis

Credit Card Market Study Interim Report: Annex 4 Switching Analysis MS14/6.2: Annex 4 Market Study Interim Report: Annex 4 November 2015 This annex describes data analysis we carried out to improve our understanding of switching and shopping around behaviour in the UK

More information

TOCILIZUMAB FOR THE TREATMENT OF RHEUMATOID ARTHRITIS

TOCILIZUMAB FOR THE TREATMENT OF RHEUMATOID ARTHRITIS TOCILIZUMAB FOR THE TREATMENT OF RHEUMATOID ARTHRITIS REPORT BY THE DECISION SUPPORT UNIT Stephen Palmer, Mark Sculpher Centre for Health Economics, University of York 6 May 2010 1 CONTENTS 1. SYNOPSIS

More information

1.0 Abstract. Title: Real Life Evaluation of Rheumatoid Arthritis in Canadians taking HUMIRA. Keywords. Rationale and Background:

1.0 Abstract. Title: Real Life Evaluation of Rheumatoid Arthritis in Canadians taking HUMIRA. Keywords. Rationale and Background: 1.0 Abstract Title: Real Life Evaluation of Rheumatoid Arthritis in Canadians taking HUMIRA Keywords Rationale and Background: This abbreviated clinical study report is based on a clinical surveillance

More information

CEIOPS-DOC-37/09. (former CP 43) October 2009

CEIOPS-DOC-37/09. (former CP 43) October 2009 CEIOPS-DOC-37/09 CEIOPS Advice for Level 2 Implementing Measures on Solvency II: Technical Provisions Article 86 f Standards for Data Quality (former CP 43) October 2009 CEIOPS e.v. Westhafenplatz 1-60327

More information

Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS)

Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS) Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS) April 30, 2008 Abstract A randomized Mode Experiment of 27,229 discharges from 45 hospitals was used to develop adjustments for the

More information

College Readiness LINKING STUDY

College Readiness LINKING STUDY College Readiness LINKING STUDY A Study of the Alignment of the RIT Scales of NWEA s MAP Assessments with the College Readiness Benchmarks of EXPLORE, PLAN, and ACT December 2011 (updated January 17, 2012)

More information

Choices in Methods for Economic Evaluation

Choices in Methods for Economic Evaluation Choices in Methods for Economic Evaluation A METHODOLOGICAL GUIDE Choices in Methods for Economic Evaluation October 2012 Department of Economics and Public Health Assessment 1 Choices in Methods for Economic

More information

PROMs risk adjustment methodology guide for general surgery and orthopaedic procedures

PROMs risk adjustment methodology guide for general surgery and orthopaedic procedures REPORT PROMs risk adjustment methodology guide for general surgery and orthopaedic procedures September 2010 In 2009, Northgate was awarded the contract to develop and apply routinely a risk adjustment

More information

Hertsmere Borough Council. Data Quality Strategy. December 2009 1

Hertsmere Borough Council. Data Quality Strategy. December 2009 1 Hertsmere Borough Council Data Quality Strategy December 2009 1 INTRODUCTION Public services need reliable, accurate and timely information with which to manage services, inform users and account for performance.

More information

Transferability of Economic Evaluations in Clinical Trials

Transferability of Economic Evaluations in Clinical Trials Transferability of Economic Evaluations in Clinical Trials Henry Glick Institutt for helseledelse og helseøkonomi November 25, 2008 The Problem Multicenter and multinational studies are the norm for the

More information

Simple Linear Regression Chapter 11

Simple Linear Regression Chapter 11 Simple Linear Regression Chapter 11 Rationale Frequently decision-making situations require modeling of relationships among business variables. For instance, the amount of sale of a product may be related

More information

National Disability Authority Resource Allocation Feasibility Study Final Report January 2013

National Disability Authority Resource Allocation Feasibility Study Final Report January 2013 National Disability Authority Resource Allocation Feasibility Study January 2013 The National Disability Authority (NDA) has commissioned and funded this evaluation. Responsibility for the evaluation (including

More information

Eliciting beliefs to inform parameters within a decision. Centre for Health Economics University of York

Eliciting beliefs to inform parameters within a decision. Centre for Health Economics University of York Eliciting beliefs to inform parameters within a decision analytic model Laura Bojke Centre for Health Economics University of York Uncertainty in decision modelling Uncertainty is pervasive in any assessment

More information

This response includes the main points that CPAG members have brought to my attention.

This response includes the main points that CPAG members have brought to my attention. Richard Jeavons Director of Commissioning Specialised Services Quarry House Leeds West Yorkshire LS2 7UB Dear Richard, Specialised Commissioning Clinical Priorities Advisory Group Skipton House 80 London

More information

NICE s Appraisal of Cancer Medications: Recent Policy and Methodological Challenges

NICE s Appraisal of Cancer Medications: Recent Policy and Methodological Challenges NICE s Appraisal of Cancer Medications: Recent Policy and Methodological Challenges Mark Sculpher, PhD Professor of Health Economics Centre of Health Economics University of York, UK Fred Hutchinson Cancer

More information

A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling

A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling A Procedure for Classifying New Respondents into Existing Segments Using Maximum Difference Scaling Background Bryan Orme and Rich Johnson, Sawtooth Software March, 2009 Market segmentation is pervasive

More information

Valuing Environmental Impacts: Practical Guidelines for the Use of Value Transfer in Policy and Project Appraisal

Valuing Environmental Impacts: Practical Guidelines for the Use of Value Transfer in Policy and Project Appraisal Valuing Environmental Impacts: Practical Guidelines for the Use of Value Transfer in Policy and Project Appraisal Technical Report Submitted to Department for Environment, Food and Rural Affairs December

More information

Econometric Analysis of Cross Section and Panel Data Second Edition. Jeffrey M. Wooldridge. The MIT Press Cambridge, Massachusetts London, England

Econometric Analysis of Cross Section and Panel Data Second Edition. Jeffrey M. Wooldridge. The MIT Press Cambridge, Massachusetts London, England Econometric Analysis of Cross Section and Panel Data Second Edition Jeffrey M. Wooldridge The MIT Press Cambridge, Massachusetts London, England Preface Acknowledgments xxi xxix I INTRODUCTION AND BACKGROUND

More information

Developing an EQ-5D-5L value set for England

Developing an EQ-5D-5L value set for England Developing an EQ-5D-5L value set for England Nancy Devlin, Ben van Hout, Koonal Shah, Brendan Mulhern, Yan Feng Patient Reported Outcomes Measures (PROMs) research conference, King s Fund, London, 3 rd

More information

OFFICIAL FILING BEFORE THE PUBLIC SERVICE COMMISSION OF WISCONSIN DIRECT TESTIMONY OF JANNELL E. MARKS

OFFICIAL FILING BEFORE THE PUBLIC SERVICE COMMISSION OF WISCONSIN DIRECT TESTIMONY OF JANNELL E. MARKS OFFICIAL FILING BEFORE THE PUBLIC SERVICE COMMISSION OF WISCONSIN Application of Northern States Power Company, a Wisconsin Corporation, for Authority to Adjust Electric and Natural Gas Rates Docket No.

More information

Adalimumab for the treatment of psoriasis

Adalimumab for the treatment of psoriasis DOI: 10.3310/hta13suppl2/07 Health Technology Assessment 2009; Vol. 13: Suppl. 2 Adalimumab for the treatment of psoriasis D Turner, J Picot,* K Cooper and E Loveman Southampton Health Technology Assessments

More information

Teriflunomide for treating relapsing remitting multiple sclerosis

Teriflunomide for treating relapsing remitting multiple sclerosis Teriflunomide for treating relapsing remitting multiple Issued: January 2014 last modified: June 2014 guidance.nice.org.uk/ta NICE has accredited the process used by the Centre for Health Technology Evaluation

More information

Social Values and Health Priority Setting Case Study

Social Values and Health Priority Setting Case Study Social Values and Health Priority Setting Case Study Title of Case Study Author Author Contact Sunitinib as a first line treatment for advanced and/or metastatic renal cell carcinoma Sarah Clark s.l.clark@ucl.ac.uk

More information

5. Multiple regression

5. Multiple regression 5. Multiple regression QBUS6840 Predictive Analytics https://www.otexts.org/fpp/5 QBUS6840 Predictive Analytics 5. Multiple regression 2/39 Outline Introduction to multiple linear regression Some useful

More information

Natalizumab for the treatment of adults with highly active relapsing remitting multiple sclerosis

Natalizumab for the treatment of adults with highly active relapsing remitting multiple sclerosis Natalizumab for the treatment of adults with highly active relapsing remitting multiple sclerosis Premeeting briefing This briefing presents major issues arising from the manufacturer s submission, Evidence

More information

What is costeffectiveness?

What is costeffectiveness? ...? series Second edition Health economics Supported by sanofi-aventis What is costeffectiveness? Ceri Phillips BSc(Econ) MSc(Econ) PhD Health Economist, Swansea University Cost-effectiveness analysis

More information

MISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group

MISSING DATA TECHNIQUES WITH SAS. IDRE Statistical Consulting Group MISSING DATA TECHNIQUES WITH SAS IDRE Statistical Consulting Group ROAD MAP FOR TODAY To discuss: 1. Commonly used techniques for handling missing data, focusing on multiple imputation 2. Issues that could

More information

New Work Item for ISO 3534-5 Predictive Analytics (Initial Notes and Thoughts) Introduction

New Work Item for ISO 3534-5 Predictive Analytics (Initial Notes and Thoughts) Introduction Introduction New Work Item for ISO 3534-5 Predictive Analytics (Initial Notes and Thoughts) Predictive analytics encompasses the body of statistical knowledge supporting the analysis of massive data sets.

More information

Composite performance measures in the public sector Rowena Jacobs, Maria Goddard and Peter C. Smith

Composite performance measures in the public sector Rowena Jacobs, Maria Goddard and Peter C. Smith Policy Discussion Briefing January 27 Composite performance measures in the public sector Rowena Jacobs, Maria Goddard and Peter C. Smith Introduction It is rare to open a newspaper or read a government

More information

Guidelines for Preparation of Review Protocols

Guidelines for Preparation of Review Protocols Guidelines for Preparation of Review Protocols Type of document: Policy _x_ Guideline Procedure Version: 1.0, January 1 2001 Decision: Steering Group, Date? A Campbell Systematic Review is meant to review

More information

Keep It Simple: Easy Ways To Estimate Choice Models For Single Consumers

Keep It Simple: Easy Ways To Estimate Choice Models For Single Consumers Keep It Simple: Easy Ways To Estimate Choice Models For Single Consumers Christine Ebling, University of Technology Sydney, christine.ebling@uts.edu.au Bart Frischknecht, University of Technology Sydney,

More information

An Alternative Route to Performance Hypothesis Testing

An Alternative Route to Performance Hypothesis Testing EDHEC-Risk Institute 393-400 promenade des Anglais 06202 Nice Cedex 3 Tel.: +33 (0)4 93 18 32 53 E-mail: research@edhec-risk.com Web: www.edhec-risk.com An Alternative Route to Performance Hypothesis Testing

More information

Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus

Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus Tihomir Asparouhov and Bengt Muthén Mplus Web Notes: No. 15 Version 8, August 5, 2014 1 Abstract This paper discusses alternatives

More information

The Concept of Project Success What 150 Australian project managers think D Baccarini 1, A Collins 2

The Concept of Project Success What 150 Australian project managers think D Baccarini 1, A Collins 2 The Concept of Project Success What 150 Australian project managers think D Baccarini 1, A Collins 2 1 Curtin University of Technology, Perth, Western Australia 2 Broad Construction Services, Perth, Western

More information

The Open University s repository of research publications and other research outputs

The Open University s repository of research publications and other research outputs Open Research Online The Open University s repository of research publications and other research outputs Using LibQUAL+ R to Identify Commonalities in Customer Satisfaction: The Secret to Success? Journal

More information

Cost-effectiveness of Pirfenidone (Esbriet ) for the treatment of Idiopathic Pulmonary Fibrosis.

Cost-effectiveness of Pirfenidone (Esbriet ) for the treatment of Idiopathic Pulmonary Fibrosis. Cost-effectiveness of Pirfenidone (Esbriet ) for the treatment of Idiopathic Pulmonary Fibrosis. March 2013 1. Pirfenidone is indicated in adults for the treatment of mild to moderate Idiopathic Pulmonary

More information

Appendix Methodology and Statistics

Appendix Methodology and Statistics Appendix Methodology and Statistics Introduction Science and Engineering Indicators (SEI) contains data compiled from a variety of sources. This appendix explains the methodological and statistical criteria

More information

5.3 SAMPLING Introduction Overview on Sampling Principles. Sampling

5.3 SAMPLING Introduction Overview on Sampling Principles. Sampling Sampling 5.3 SAMPLING 5.3.1 Introduction Data for the LULUCF sector are often obtained from sample surveys and typically are used for estimating changes in land use or in carbon stocks. National forest

More information

Multiple Imputation for Missing Data: A Cautionary Tale

Multiple Imputation for Missing Data: A Cautionary Tale Multiple Imputation for Missing Data: A Cautionary Tale Paul D. Allison University of Pennsylvania Address correspondence to Paul D. Allison, Sociology Department, University of Pennsylvania, 3718 Locust

More information

Appendix E : Evidence table 7b Patients view s and experiences of falls prevention strategies: Quantitative studies

Appendix E : Evidence table 7b Patients view s and experiences of falls prevention strategies: Quantitative studies Specific falls prevention programs or general behaviour change interventions Culos-Reed 2000 Lambert 2001 Yardley 2002 Narrative review of predictors of adherence to behaviour change interventions. Before/after

More information

Assessment Policy. 1 Introduction. 2 Background

Assessment Policy. 1 Introduction. 2 Background Assessment Policy 1 Introduction This document has been written by the National Foundation for Educational Research (NFER) to provide policy makers, researchers, teacher educators and practitioners with

More information

UCB. Certolizumab pegol (CIMZIA ) for the treatment of Rheumatoid Arthritis PATIENT ACCESS SCHEME (PAS) SUBMISSION TO NICE

UCB. Certolizumab pegol (CIMZIA ) for the treatment of Rheumatoid Arthritis PATIENT ACCESS SCHEME (PAS) SUBMISSION TO NICE UCB Certolizumab pegol (CIMZIA ) for the treatment of Rheumatoid Arthritis PATIENT ACCESS SCHEME (PAS) SUBMISSION TO NICE July 23 d 2009 1 Executive summary UCB have proposed a patient access scheme (PAS)

More information

ASSESSING TECHNOLOGIES AT THE END OF LIFE: A REVIEW REPORT BY THE DECISION SUPPORT UNIT OF EMPIRICAL EVIDENCE.

ASSESSING TECHNOLOGIES AT THE END OF LIFE: A REVIEW REPORT BY THE DECISION SUPPORT UNIT OF EMPIRICAL EVIDENCE. ASSESSING TECHNOLOGIES AT THE END OF LIFE: A REVIEW OF EMPIRICAL EVIDENCE. REPORT BY THE DECISION SUPPORT UNIT Simon Dixon, Louise Longworth, Allan Wailoo School of Health and Related Research, University

More information

Marketing Mix Modelling and Big Data P. M Cain

Marketing Mix Modelling and Big Data P. M Cain 1) Introduction Marketing Mix Modelling and Big Data P. M Cain Big data is generally defined in terms of the volume and variety of structured and unstructured information. Whereas structured data is stored

More information

11/20/2014. Correlational research is used to describe the relationship between two or more naturally occurring variables.

11/20/2014. Correlational research is used to describe the relationship between two or more naturally occurring variables. Correlational research is used to describe the relationship between two or more naturally occurring variables. Is age related to political conservativism? Are highly extraverted people less afraid of rejection

More information

Canonical Correlation Analysis

Canonical Correlation Analysis Canonical Correlation Analysis LEARNING OBJECTIVES Upon completing this chapter, you should be able to do the following: State the similarities and differences between multiple regression, factor analysis,

More information

Well-being and the value of health

Well-being and the value of health Well-being and the value of health Happiness and Public Policy Conference Bangkok, Thailand 8-9 July 2007 Bernard van den Berg Department of Health Economics & Health Technology Assessment, Institute of

More information

Models for Product Demand Forecasting with the Use of Judgmental Adjustments to Statistical Forecasts

Models for Product Demand Forecasting with the Use of Judgmental Adjustments to Statistical Forecasts Page 1 of 20 ISF 2008 Models for Product Demand Forecasting with the Use of Judgmental Adjustments to Statistical Forecasts Andrey Davydenko, Professor Robert Fildes a.davydenko@lancaster.ac.uk Lancaster

More information

Location matters. 3 techniques to incorporate geo-spatial effects in one's predictive model

Location matters. 3 techniques to incorporate geo-spatial effects in one's predictive model Location matters. 3 techniques to incorporate geo-spatial effects in one's predictive model Xavier Conort xavier.conort@gear-analytics.com Motivation Location matters! Observed value at one location is

More information

Chapter 10. Key Ideas Correlation, Correlation Coefficient (r),

Chapter 10. Key Ideas Correlation, Correlation Coefficient (r), Chapter 0 Key Ideas Correlation, Correlation Coefficient (r), Section 0-: Overview We have already explored the basics of describing single variable data sets. However, when two quantitative variables

More information

Survival Analysis Using SPSS. By Hui Bian Office for Faculty Excellence

Survival Analysis Using SPSS. By Hui Bian Office for Faculty Excellence Survival Analysis Using SPSS By Hui Bian Office for Faculty Excellence Survival analysis What is survival analysis Event history analysis Time series analysis When use survival analysis Research interest

More information

Evidence Based Practice Information Sheets for Health Professionals. An Introduction to Systematic Reviews. This Sheet Covers the Following Concepts:

Evidence Based Practice Information Sheets for Health Professionals. An Introduction to Systematic Reviews. This Sheet Covers the Following Concepts: Supplement 1, 2001 ChangingPractice Evidence Based Practice Information Sheets for Health Professionals An Introduction to Systematic Reviews The series Changing Practice has been designed to support health

More information

Lean Six Sigma Training/Certification Book: Volume 1

Lean Six Sigma Training/Certification Book: Volume 1 Lean Six Sigma Training/Certification Book: Volume 1 Six Sigma Quality: Concepts & Cases Volume I (Statistical Tools in Six Sigma DMAIC process with MINITAB Applications Chapter 1 Introduction to Six Sigma,

More information

City University London School of Community & Health Sciences. Assessment Specification

City University London School of Community & Health Sciences. Assessment Specification Programme(s) Module Code(s) Curriculum Theme Module Title Assessment Type Type Submission Deadline City University London School of Community & Health Sciences Assessment Specification BSc (Hons) Adult

More information

Good Practice in. Educational Research Writing

Good Practice in. Educational Research Writing Good Practice in Educational Research Writing 1 1. PREAMBLE 1.1 There are two main thrusts to educational research, viz: to inform understandings of educational issues, drawing on and developing educational

More information

Introduction to Statistics for Computer Science Projects

Introduction to Statistics for Computer Science Projects Introduction Introduction to Statistics for Computer Science Projects Peter Coxhead Whole modules are devoted to statistics and related topics in many degree programmes, so in this short session all I

More information

REVIEW OF NEW UNIVERSITY-WIDE TEACHING EVALUATION FORM

REVIEW OF NEW UNIVERSITY-WIDE TEACHING EVALUATION FORM Excerpt from the PACTL Report to Senate of May 15, 1998 (S.98-123) Page 1 Provost's Advisory Committee on Teaching and Learning (PACTL) Report to Senate REVIEW OF NEW UNIVERSITY-WIDE TEACHING EVALUATION

More information

Psychological therapies for borderline personality disorder. Health Technology Assessment 2006; Vol. 10: No. 35

Psychological therapies for borderline personality disorder. Health Technology Assessment 2006; Vol. 10: No. 35 Psychological therapies for borderline personality disorder Psychological therapies including dialectical behaviour therapy for borderline personality disorder: a systematic review and preliminary economic

More information

A developmental framework for pharmacists progressing to advanced levels of practice

A developmental framework for pharmacists progressing to advanced levels of practice ACLF Advanced to Consultant level Framework A developmental framework for pharmacists progressing to advanced levels of practice Version 2009(a) CoDEG www.codeg.org ADVANCED AND CONSULTANT LEVEL COMPETENCY

More information

Methods for Meta-analysis in Medical Research

Methods for Meta-analysis in Medical Research Methods for Meta-analysis in Medical Research Alex J. Sutton University of Leicester, UK Keith R. Abrams University of Leicester, UK David R. Jones University of Leicester, UK Trevor A. Sheldon University

More information

Justifying the adequacy of samples in qualitative interview-based studies: Differences between and within journals

Justifying the adequacy of samples in qualitative interview-based studies: Differences between and within journals Justifying the adequacy of samples in qualitative interview-based studies: Differences between and within journals Prof Julie Barnett a, Konstantina Vasileiou a, Dr Susan Thorpe b, Prof Terry Young c a

More information

Decision Support Unit Project Specification Form

Decision Support Unit Project Specification Form Decision Support Unit Project Specification Form Project Number Topic Synopsis of the technical issue DSU/2014-15/M/21 Development of a transparent decision making framework for routine use by appraisal

More information

F. Farrokhyar, MPhil, PhD, PDoc

F. Farrokhyar, MPhil, PhD, PDoc Learning objectives Descriptive Statistics F. Farrokhyar, MPhil, PhD, PDoc To recognize different types of variables To learn how to appropriately explore your data How to display data using graphs How

More information

Schools Value-added Information System Technical Manual

Schools Value-added Information System Technical Manual Schools Value-added Information System Technical Manual Quality Assurance & School-based Support Division Education Bureau 2015 Contents Unit 1 Overview... 1 Unit 2 The Concept of VA... 2 Unit 3 Control

More information

Adequacy of Biomath. Models. Empirical Modeling Tools. Bayesian Modeling. Model Uncertainty / Selection

Adequacy of Biomath. Models. Empirical Modeling Tools. Bayesian Modeling. Model Uncertainty / Selection Directions in Statistical Methodology for Multivariable Predictive Modeling Frank E Harrell Jr University of Virginia Seattle WA 19May98 Overview of Modeling Process Model selection Regression shape Diagnostics

More information

EBM Cheat Sheet- Measurements Card

EBM Cheat Sheet- Measurements Card EBM Cheat Sheet- Measurements Card Basic terms: Prevalence = Number of existing cases of disease at a point in time / Total population. Notes: Numerator includes old and new cases Prevalence is cross-sectional

More information

Common resource allocation framework

Common resource allocation framework Common resource allocation framework Financial framework Introduction This document outlines the purpose of the financial framework, its principles and the process for developing the financial elements

More information

Loss Given Default models for UK retail credit cards

Loss Given Default models for UK retail credit cards Loss Given Default models for UK retail credit cards Tony Bellotti and Jonathan Crook Credit Research Centre University of Edinburgh Business School William Robertson Building 50 George Square, Edinburgh

More information

University of Oklahoma

University of Oklahoma A peer-reviewed electronic journal. Copyright is retained by the first or sole author, who grants right of first publication to Practical Assessment, Research & Evaluation. Permission is granted to distribute

More information

STATISTICA Formula Guide: Logistic Regression. Table of Contents

STATISTICA Formula Guide: Logistic Regression. Table of Contents : Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

More information

Checklist for Quasi-Experimental Studies (non-randomized experimental studies)

Checklist for Quasi-Experimental Studies (non-randomized experimental studies) The Joanna Briggs Institute Critical Appraisal tools for use in JBI Systematic Reviews Checklist for Quasi-Experimental Studies (non-randomized experimental studies) http://joannabriggs.org/research/critical-appraisal-tools.html

More information

Fixed-Effect Versus Random-Effects Models

Fixed-Effect Versus Random-Effects Models CHAPTER 13 Fixed-Effect Versus Random-Effects Models Introduction Definition of a summary effect Estimating the summary effect Extreme effect size in a large study or a small study Confidence interval

More information

Inferential Statistics

Inferential Statistics Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

More information

18.6.1 Terms concerned with internal quality control procedures

18.6.1 Terms concerned with internal quality control procedures 18.6.1 Terms concerned with internal quality control procedures Quality assurance in analytical laboratories Quality assurance is the essential organisational infrastructure that underlies all reliable

More information

Survey Process White Paper Series The Six Steps in Conducting Quantitative Marketing Research

Survey Process White Paper Series The Six Steps in Conducting Quantitative Marketing Research Survey Process White Paper Series The Six Steps in Conducting Quantitative Marketing Research POLARIS MARKETING RESEARCH, INC. 1455 LINCOLN PARKWAY, SUITE 320 ATLANTA, GEORGIA 30346 404.816.0353 www.polarismr.com

More information

HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION

HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION HOD 2990 10 November 2010 Lecture Background This is a lightning speed summary of introductory statistical methods for senior undergraduate

More information

Data Analysis and Sampling

Data Analysis and Sampling Data Analysis and Sampling About This Course Course Description In order to perform successful internal audits, you must know how to reduce a large data set down to critical subsets based on risk or importance,

More information

Yiming Peng, Department of Statistics. February 12, 2013

Yiming Peng, Department of Statistics. February 12, 2013 Regression Analysis Using JMP Yiming Peng, Department of Statistics February 12, 2013 2 Presentation and Data http://www.lisa.stat.vt.edu Short Courses Regression Analysis Using JMP Download Data to Desktop

More information

Multiple Regression: What Is It?

Multiple Regression: What Is It? Multiple Regression Multiple Regression: What Is It? Multiple regression is a collection of techniques in which there are multiple predictors of varying kinds and a single outcome We are interested in

More information

Experimental data and survey data

Experimental data and survey data Experimental data and survey data An experiment involves the collection of measurements or observations about populations that are treated or controlled by the experimenter. A survey is an examination

More information

Prediction and Confidence Intervals in Regression

Prediction and Confidence Intervals in Regression Fall Semester, 2001 Statistics 621 Lecture 3 Robert Stine 1 Prediction and Confidence Intervals in Regression Preliminaries Teaching assistants See them in Room 3009 SH-DH. Hours are detailed in the syllabus.

More information

Genetic Discoveries and the Role of Health Economics

Genetic Discoveries and the Role of Health Economics Genetic Discoveries and the Role of Health Economics Collaboration for Outcome Research and Evaluation (CORE) Faculty of Pharmaceutical Sciences University of British Columbia February 02, 2010 Content

More information

National Child Measurement Programme: England, 2011/12 school year

National Child Measurement Programme: England, 2011/12 school year National Child Measurement Programme: England, 2011/12 school year December 2012 Copyright 2012, The Health and Social Care Information Centre. All Rights Reserved. www.ic.nhs.uk Author: The Health and

More information

Erlotinib for the treatment of non-small cell lung cancer. Report from the Decision Support Unit in response to additional data submitted by Roche

Erlotinib for the treatment of non-small cell lung cancer. Report from the Decision Support Unit in response to additional data submitted by Roche Erlotinib for the treatment of non-small cell lung cancer Report from the Decision Support Unit in response to additional data submitted by Roche 10 th January 2008 Part 1: A Sutton. Critique of the network

More information

Problems with OLS Considering :

Problems with OLS Considering : Problems with OLS Considering : we assume Y i X i u i E u i 0 E u i or var u i E u i u j 0orcov u i,u j 0 We have seen that we have to make very specific assumptions about u i in order to get OLS estimates

More information

Handling attrition and non-response in longitudinal data

Handling attrition and non-response in longitudinal data Longitudinal and Life Course Studies 2009 Volume 1 Issue 1 Pp 63-72 Handling attrition and non-response in longitudinal data Harvey Goldstein University of Bristol Correspondence. Professor H. Goldstein

More information

Highways Maintenance Capital Funding Self-Assessment Questionnaire for the Incentive Fund 1 Important information

Highways Maintenance Capital Funding Self-Assessment Questionnaire for the Incentive Fund 1 Important information Highways Maintenance Capital Funding Self-Assessment Questionnaire for the Incentive Fund 1 Important information What is the purpose of this questionnaire? In December 2014, the Secretary of State for

More information

PERFORMANCE & PERSONAL DEVELOPMENT PROGRAMME Launched: April 2010

PERFORMANCE & PERSONAL DEVELOPMENT PROGRAMME Launched: April 2010 PERFORMANCE & PERSONAL DEVELOPMENT PROGRAMME Launched: April 2010 1 PERFORMANCE & PERSONAL DEVELOPMENT PROGRAMME INDEX Page Introduction 3 PPDP Cycle 4 Performance planning meeting 5 Performance planning

More information

Trade-Off Study Sample Size: How Low Can We go?

Trade-Off Study Sample Size: How Low Can We go? Trade-Off Study Sample Size: How Low Can We go? The effect of sample size on model error is examined through several commercial data sets, using five trade-off techniques: ACA, ACA/HB, CVA, HB-Reg and

More information

When Does it Make Sense to Perform a Meta-Analysis?

When Does it Make Sense to Perform a Meta-Analysis? CHAPTER 40 When Does it Make Sense to Perform a Meta-Analysis? Introduction Are the studies similar enough to combine? Can I combine studies with different designs? How many studies are enough to carry

More information

American Association for Laboratory Accreditation

American Association for Laboratory Accreditation Page 1 of 12 The examples provided are intended to demonstrate ways to implement the A2LA policies for the estimation of measurement uncertainty for methods that use counting for determining the number

More information