Big data size isn t enough! Irene Petersen, PhD Primary Care & Population Health

Transcription

1 Big data size isn t enough! Irene Petersen, PhD Primary Care & Population Health

2 Introduction Reader (Statistics and Epidemiology) Research team epidemiologists/statisticians/phd students Primary care databases THIN and CPRD studies Research topics Prescribed medicine in pregnancy Mental health Methodological questions Missing data Regression Discontinuity Design (RDD) Confounding (by indication) Cardiovascular diseases Infectious diseases Or just google THIN UCL

3 Today Big data - Electronic Health Records Safety and efficacy of medicine in real life Confounding and selection bias Potential solutions

4 Big Data - Electronic Health Records Primary Care Databases THIN, CPRD, QRESEARCH Administrative Databases Insurance claims databases Hospital Episodes Statistics Population registers Scandinavian registers

5 UK primary care databases (1) THIN, CPRD, Qresearch Anonymised records million years of patient data Medical diagnoses and symptoms, preventative measures, test results and immunisations, prescriptions, referrals to secondary care and free text information

6 UK primary care databases (2) Demographic information e.g. year of birth, sex, social deprivation (Townsend score) Broadly representative of the UK population (sex, age, size of practice and geographic distribution) For clinical management NOT for research

7 Electronic Health Records versus Randomised Controlled Trials Electronic Health Records Randomised Controlled Trials Data collection Clinical sessions At fixed time points Data Missing data Coded clinical records Read or ICD-10, measurements Well people have less data Interviews, questionnaires, measurements Random(?) Size Millions From hundreds to thousands Treatment Selective Randomised

8 Recording of weight by age and gender in THIN Age (years) Male Female

9 Recording of weight in diabetics and nondiabetics 80 measurement recorded weight Year measurement recorded Registered 1995 Registered 2000 Registered 2005 Registered 2010 solid line - diabetes, dashed line - no diabetes

10 Strength of Electronic Health Records Large sample sizes Population that is often difficult to follow up by other means Very elderly Pregnant women People with severe mental illnesses Real life data Long follow-up

11 Provide some fantastic opportunities! Explore effectiveness in populations not covered by RCTs Very elderly, pregnant, people with severe mental illnesses Examine adverse (drug) effects Bridge from RCT to real life

12 Before we get too excited. Let s look at some of the challenges and pitfalls

13 Case-control or Cohort study? Drug A is commonly prescribed Outcome B is rare Use electronic health records to address the question

14 88,125 cases and 362,254 matched controls Cases between 30 and 100 years 5 matched controls to be alive at the date of cancer Adjusted analysis for potential confounders such as smoking and social deprivation and some specific diseases

15 Prolonged use of statin (more than 4 years) Associated with significantly increased risk of: Colorectal cancer (OR 1.23, 95%CI 1.10 to 1.38) Bladder cancer (OR 1.29, 95%CI 1.08 to 1.54) Lung cancer (OR 1.18, 95%CI 1.05 to 1.34)

16 Hang on. Let s have a look at this again Cases were those who got cancer Controls were a random sample of those who were alive at the time the cases got cancer What if..

17 Those who receive prolonged statin treatment More likely to die from other causes than cancer (e.g. cardiovascular diseases)? Then they would be less likely to enter the control group

18 Case-control study with live controls Estimates sensitive to difference in survival rates between exposed and unexposed

19 Interpretation of case-control study cohort study Case-control study: Of the survivors what is the chance that those with cancer were exposed to statin? Cohort study: Of those exposed to statin what is the risks of developing cancer compared to those not exposed

20 Cohort study found no associations between statins and cancers Smeeth et al. British Journal of Clinical Pharmacology

21 Let s look at a few other examples

22 High dose antidepressants increase self harm in young people? Miller et al. JAMA Internal Medicine, April 2014

23 High dose antidepressants increase self harm in young people? Propensity score matched cohort study Health care utilization data from US residents Depression ages 10 to 64 years who initiated antidepressant therapy Standard (modal) versus higher doses

24 Propensity score methods Propensity Scores (PS) estimate the predicted probability (propensity) of use of a given drug PS based on his/her characteristics when the treatment is chosen logistic treatment sex age x1 x2 x3 x4 x5 x6 predict Predicted value (between 0-1) is the propensity score

25 Propensity scores Sturmer et al. J Clin Epidemiol May ; 59(5): Williamson et al. Statistical Methods in Medical Research (3)

26 High dose antidepressants increase self harm in young people? High versus modal dose in young people: hazard ratio [HR], 2.2 [95%CI, ] 1 additional event for every 150 such patients treated with high-dose (instead of modal-dose) therapy High versus model dose in adults (25 64 years): HR, 1.2 [95%CI, ]

27 Conclusions and implication Children and young adults. at hightherapeutic (rather than modal-therapeutic) doses seem to be at heightened risk of deliberate self-harm. Our findings offer clinicians an additional incentive to avoid initiating pharmacotherapy at high-therapeutic doses.

28

29 What is the problem?

30 Health care data Randomised Controlled Trial Propensity score methods may balance measured characteristics but. Propensity score methods may NOT balance unmeasured characteristics

31 Not just random allocation Clinicians make a treatment decision Acute presentation of psychiatric illness Severe ill individuals may receive higher doses This information is NOT captured in the health care databases

32 Confounding by indication Acute and severe psychiatric problem High dose antidepressant self harm

33 Confounding by indication Acute and severe psychiatric problem High dose antidepressant self harm

34 High dose antidepressant A mere marker of acute severe psychiatric problems Underlying condition associated with increased self-harm? Too early to conclude that high doses of antidepressants are unsafe Leave young people without treatment!?

35 Efficacy of heart failure treatment in real life? An example from our own work

36 Spironolactone treatment for heart failure Spironolactone improve survival in people with heart failure in Randomised Controlled Trials What about in real life?

37 RALES trial - NEJM September 1999 The trial was discontinued early, after 24 months Spironolactone was efficacious 386 deaths in the placebo group (46%) 284 in the spironolactone group (35%) Relative risk of death: Hazard Ratio: 0.70 (95% CI, 0.60 to 0.82; P<0.001)

38 Bridge from RCT to real life Identified people with severe heart failure in THIN Propensity score for spironolactone treatment Matched individuals with and without treatment Estimated relative survival using a Cox model

39 What did we find? People treated with spironolactone had same chance of survival after 24 months as those treated in RALES trial People with heart failure, but NOT treated, had an even better survival!

40 Relative risk of mortality in people treated with spironolactone RALES: Hazard Ratio: 0.70 (95% CI ) Our study: Hazard Ratio: 1.32 (95% CI ) Freemantle et al. BMJ 2013

41 Similar problem as before. Treatment is NOT randomly allocated Doctors make a choice Spironolactone given to those with worse prognosis More severe heart failure Acute situation? Reason for prescribing was NOT recorded Propensity score cannot solve the issue of confounding by indication

42

43 Bridge from Randomised Controlled Trials to real life - Statin example Propensity score matched sample This time, they were able to replicate trial results!

44 Why was it possible in this situation? Statin was used as primary prevention, rather than treatment of illness Potential confounders were captured in data Weight, blood pressure, sex, deprivation etc.

45 Propensity Scores and other regression adjustments may. It may work when Treatment is given as prevention and data is recorded in the database (e.g. statin) Decision to treat unrelated to prognosis of outcome (unexpected effects) It may NOT work when treatment is given in response to acute clinical need or people who are more frail (e.g.antidepressant, spironolactone, hypnotics)

46 The future for analysis and design of BIG data studies Cohort rather case-control studies Still need to think design through carefully Cohort studies including Several comparison groups Active comparisons e.g. drug versus drug Sequential simulated trials Self-controlled case series (SCCS) Instrumental Variables Regression Discontinuation Design

47 The future for analysis and design of BIG data studies (2) Sequential Simulated trials Run series of trials in database Trial 1: those initiated on treatment in 2001 versus those not initiated in 2001 Trial 2: those initiated on treatment in 2002 versus those not initiated in 2002 Danaei et al Toh & Manson 2013

48 The future for analysis and design of BIG data studies (3) Sequential Simulated trials Avoid selection based on the future Less likely to have a healthy user or healthy non-user effect Makes is easy to define start for non-exposed

49 The future for analysis and design of BIG data studies (4) Self-controlled case series (SCCS) methodology Use individuals as their own control See

50 The future for analysis and design of BIG data studies (4) Instrumental variables Use variation among general practices?

51 Summary Electronic Health Records reflect real life clinical practice Randomised Controlled Trials Confounding (by indication) and selection bias still are major issues! - think about the study design Interpretation of results Offer more than one interpretation perhaps drug is just a marker? Accept the limitations of BIG data

52

53 References 1. Freemantle N, Marston L, Walters K, Wood J, Reynolds MR, Petersen I. Making inferences on treatment effects from real world data: propensity scores, confounding by indication, and other perils for the unwary in observational research. BMJ Nov 11;347(nov11 3):f6409 f Williamson E, Morley R, Lucas A, Carpenter J. Propensity scores: From naive enthusiasm to intuitive understanding. Stat Methods Med Res Jun 1;21(3): Stürmer T, Joshi M, Glynn RJ, Avorn J, Rothman KJ, Schneeweiss S. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol May;59(5): Rosenbaum PR, Rubin DB. The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika Apr;70(1): Miller M, Swanson SA, Azrael D, Pate V, Stürmer T. Antidepressant Dose, Age, and the Risk of Deliberate Self-harm. JAMA Intern Med [Internet] Apr 28 [cited 2014 May 1]; Available from: 6. Smeeth L, Douglas I, Hall AJ, Hubbard R, Evans S. Effect of statins on a wide range of health outcomes: a cohort study validated by comparison with randomized trials. Br J Clin Pharmacol Jan;67(1): Toh S, Manson JE. An Analytic Framework for Aligning Observational and Randomized Trial Data: Application to Postmenopausal Hormone Therapy and Coronary Heart Disease. Stat Biosci Nov;5(2): Whitaker HJ, Paddy Farrington C, Spiessens B, Musonda P. Tutorial in biostatistics: the self-controlled case series method. Stat Med. 2006;25(10):