The Consequences of Missing Data in the ATLAS ACS 2-TIMI 51 Trial In this white paper, we will explore the consequences of missing data in the ATLAS ACS 2-TIMI 51 Trial and consider if an alternative approach to treating missing data could have been adopted.
Introduction ATLAS ACS 2-TIMI 51 is a Phase III trial that evaluated rivaroxaban (a blood-thinning agent) as an adjunctive therapy in patients with Acute Coronary Syndrome (ACS) with the aim of determining a clinically effective low-dose regimen. The trial demonstrated a significant reduction in a composite of cardiovascular death, myocardial infraction and stroke, thus meeting its primary efficacy endpoint. However, in May 2012, in an unexpected decision, an FDA Advisory Panel voted against recommending the approval of rivaroxaban for an expanded ACS indication. Concerns about the extent and treatment of missing data in the ATLAS ACS 2-TIMI 51 Trial were cited by reviewers as one of the reasons for this surprise outcome. In this white paper, we will explore the consequences of missing data in the ATLAS ACS 2-TIMI 51 Trial and consider if an alternative approach to treating missing data could have been adopted. Brief Overview of the ATLAS ACS 2-TIMI 51 Design and Results The ATLAS ACS 2-TIMI 51 trial was conducted from November 2008 through to September 2011. The trial design can be summarized as follows: Randomized, double-blind, placebo-controlled, event driven study Evaluate the efficacy and safety of rivaroxaban Sample size 15,526 patients Three arm trial: o 2.5mg dose o 5mg dose o Placebo Primary evaluation strategy: mitt Primary objective: demonstrate superiority of rivaroxaban (at each dose level) compared to placebo in reducing cardiovascular death, MI and stroke. Headline results were very positive. Rivaroxaban significantly reduced the primary efficacy end point of death from cardiovascular cause, myocardial infraction or stroke as compared to placebo with rates of 8.9% and 10.7% respectively (1). Looking closer at the two doses of Rivaroxaban under consideration in this trial, each of the doses reduced the primary efficacy endpoint compared to placebo; the 2.5mg dose by 9.1% and the 5mg dose by 8.8% as compared to placebo rate of 10.7%. Each dose level of Rivaroxaban did however increase incidents of both major and minor bleeding, but with no significant increase in fatal bleeding with either dose. The rates of other adverse events, not related to bleeding were similar across Rivaroxaban and placebo groups. Missing Data in the ATLAS ACS 2-TIMI 51 According to the ATLAS ACS 2- TIMI 51 Clinical Protocol, no
imputation procedure would be applied in cases of missing data (2). 2402 patients (15.5%) prematurely discontinued from the study with 1,294 patients (8.3%) withdrawing consent. This level of missing data and its impact on the overall interpretability of the trial results continues to cause serious concerns for FDA reviewers. On January 16 th 2014, the FDA advisory panel again voted against recommending Rivaroxaban for the treatment of ACS in patients who had previously experienced a heart attack. Panelist Steven Nissen remarked, It s not just that the data are fragile, it s that the therapy has both benefits and harms and in that context the quality of the data becomes increasingly important (3). Another panel member, Linda Fried concluded that while the ATLAS ACS 2-TIMI 51 was probably an overall positive study, because of the large amount of missing data and the absence of a confirmatory trial, it was not robust enough to support the ACS indication (3). Furthermore, at the earlier panel hearing in May 2012, Dr. Scott Emerson, a statistician at the University of Washington Seattle was concerned about how the ATLAS ACS 2-TIMI 51 data would stand up under missing data sensitivity analysis. He commented Differential event rates after dropout are the number-one thing we re afraid of, so you have to explore it. It would not surprise me if, at the end of the day, these data did not hold up under a proper sensitivity analysis (4). Although there are no regulatory guidelines that stipulate an acceptable level of missing-ness or an acceptable method to handle missing data, a recently published NRC report entitled The Prevention and Treatment of Missing Data in Clinical Trials (5) does offer some guidance to researchers as to what is no longer appropriate and what will be more acceptable to regulatory agencies in the future. Some researchers believe that this report is a precursor to a long-awaited FDA Guidance document on handling missing data. Some of the key recommendations on handling missing data made in this NCR report are as follows: Recommendation 10: Single imputation methods like last observation carried forward (LOCF) and baseline observation carried forward should not be used as the primary approach to the treatment of missing data unless the assumptions that underlie them are scientifically justified. Recommendation 11: Parametric models in general, and random effects models in particular, should be used with caution, with all their assumptions clearly spelled out and justified. Models relying on parametric assumptions should be accompanied by goodness-of-fit procedures. Recommendation 12: It is important that the primary analysis of the data from a clinical trial should account for the uncertainty attributable to missing data, so that under the stated missing data assumptions the associated
significance tests have valid type I error rates and the confidence intervals have the nominal coverage properties. For inverse probability weighting and maximum likelihood methods, this analysis can be accomplished by appropriate computation of standard errors, using either asymptotic results or the bootstrap. For imputation, it is necessary to use appropriate rules for multiply imputing missing responses and combining results across imputed datasets because single imputation does not account for all sources of variability. Recommendation 15: Sensitivity analyses should be part of the primary reporting of findings from clinical trials. Examining sensitivity to the assumptions about the missing data mechanism should be a mandatory component of reporting. While the NRC report is very clear in its assertion that prevention of missingness rather than treatment remains the optimal approach to limit missing data problems, it is also obvious from this report that traditional methods of handling missing data such as LOCF or the option taken in the ATLAS ACS 2- TIMI 51 trial of simply ignoring missing data are no longer appropriate. Other modern methods such as multiple imputation and effective sensitivity analysis are now required to ensure that the impact of missing data on large phase III trials does not jeopardize the investments of time and resources devoted to studies such as ATLAS ACS 2-TIMI 51. Multiple Imputation of Missing Data in Clinical Trials First proposed by Rubin in the 1970 s, the method imputes several values (M) for each missing value, thereby deriving estimates of uncertainty that incorporate the added variance due to missing data (6). Analytical incorporation of the uncertainty due to missing data is highly desirable as it helps to preserve the p-value estimates and gives greater accuracy to subsequent data analysis and increases the validity of trial results. Including the use of multiple imputation as your missing data method in your clinical protocol will help to enhance the credibility of causal inferences from clinical trials. Benefits of multiple imputation include: Add validity & credibility to your clinical trial protocol by handling missing data in an effective, scientific and recommended manner. Preserve p-value estimates. Demonstrate the robustness of data outcomes by complying with sensitivity analysis requirements. With recent advances in computing power over the past decade, multiple imputation has now become a real option for use on clinical trial data. The development of a number of commercially available software applications dedicated to the technique has made it even easier to specify multiple imputation in your protocol documentation.
Software for Multiple Imputation SOLAS for Missing Data Analysis, developed with guidance from Prof Donald Rubin is a comprehensive software solution designed to help researchers analyze data with missing values. SOLAS for Missing Data Analysis offers authority and validity to your clinical trial. Thanks to its 9 different imputation methods, SOLAS for Missing Data Analysis can easily perform sensitivity analysis so that you can fully stress test your missing data assumptions. It is the most complimentary product to use with SAS for missing data analysis because of the SOLAS/SAS Data Transfer, which allows users to seamlessly import and export to and from SAS with an easy to use interface; as well as save in SAS file format eliminating any file corruption issues. SOLAS for Missing Data Analysis also provides users with unique graphs to help quickly identify missing data, key drivers of missingness, relationships and patterns. These provide instant understanding and insight early on in the missing data analysis process. For more information please visit www.statsols.com/products/solas References 1. Mega JL, Braunwald EB, Wivioti SD, Bassand JP, Bhatt DL, Bode C, et al: Rivaroxaban in Patients with Recent Acute Coronary Syndrome, N Engl J Med 2012 366:9-10. 2. Krantz MJ, Kaul S: The Atlas ACS 2-TIMI 51 Trial and the Burden of Missing Data, Journal of the American College of Cardiology Vol 62, No.9 2013. 3. FirstWord Pharma FDA Advisory Panel recommends against expanded approval for Johnson & Johnson s Xarelto Jan 14 th 2014. 4. Medscape. May 23, 2012. Missing Data Lead FDA Panel to Vote Against Rivaroxaban for ACS. 5. National Research Council: The Prevention and Treatment of Missing Data in Clinical Trials. Washington, DC: The National Academies Press, 2010. 6. Little R, Rubin D: Statistical Analysis with Missing Data 2 nd Edition: Wiley & Sons New York 2002. 7. U.S. Food and Drug Administration May 23, 2012, Meeting of the Cardiovascular and Renal Drugs Advisory Committee. Available at: www.fda.gov/advisorycommitte es/committeesmeetingmateials/ Drugs/CardiovascarandRenalDr ugsadvisorycommittee/ucm285 415.htm. Accessed January 16 2014. 8. Food and Drug Administration (2008) Guidance for Sponsors, Clinical Investigators, and IRBs: Data Retention When Subjects Withdraw from FDA-Regulated Clinical Trials.