Overview of ICH E9: Statistical Principles for Clinical Trials Mario Chen Family Health International Biostatistics Workshop India, March 2007
Scope and Direction Focus on statistical principles Gives direction to researchers in design, conduct, analysis, and evaluation of trials Does not address use of specific statistical tests Emphasis on later phase, confirmatory trials Target audience: individuals from a broad range of scientific disciplines Statisticians, clinicians, pharmacologists, epidemiologists
Scope and Direction Trial Statistician: Responsible for all the statistical work associated with the trial Ensures statistical principles are appropriately applied Has the proper training and experience to implement the principles in this guidance
Scope and Direction Minimizing bias Systematic tendency of any factors associated with design, conduct, analysis and interpretation to lead to an estimate of treatment effect different from the true value. Maximizing precision Obtaining small standard errors and narrow confidence intervals. Evaluating robustness Sensitivity of overall conclusions to various limitations of the data, assumptions, analysis procedures used.
Scope and Direction Controlling the type I error Ensuring that the chance of declaring a treatment efficacious when it in fact does not work is low (e.g., α 0.05) Multiplicity refers to having more than one opportunity to detect a difference between drugs (e.g., interim analyses, multiple endpoints of interest)
Protocol and Analysis Plan Principal features of statistical analysis should be clearly specified in the protocol Protocol (and amendments) should be approved by trained statistician A detailed Analysis Plan should be written before data analysis begins
Types of Trials Confirmatory Trial Exploratory Trial
Confirmatory Trial An adequately controlled trial in which the hypotheses are stated in advance and evaluated. Key hypothesis of interest follows directly from the trial s primary objective is always pre-defined is the hypothesis that is subsequently tested when the trial is complete Adherence to protocols and SOPs is particularly important
Exploratory Trial Clear and precise objectives, however, tests of hypothesis may be data dependent Such trials cannot be the basis of the formal proof of efficacy
Population Earlier phases may focus on a very narrow subgroup Confirmatory trials should more closely mirror target population of the therapy under study Issues of Generalizability Clear Inclusion/Exclusion criteria
Outcome Variables Primary variable(s) directly related to the primary objective preferable to specify only one reliable and validated variable used in earlier studies or in published literature used when estimating the sample size Secondary variables either supportive measurements related to the primary objective, or measurements of effects related to the secondary objectives
Avoiding Bias by Design: Blinding Blinding limits the occurrence of conscious and unconscious bias arising from the influence one s knowledge of treatment may have on recruitment and allocation of subjects their subsequent care attitudes of subjects to the treatments assessment of endpoints handling of withdrawals exclusion of data from analysis choice of analysis methods
Avoiding Bias by Design: Randomization introduces a deliberate element of chance into the assignment of treatments provides a sound statistical basis for the comparison of treatment groups tends to produce treatment groups with distributions of prognostic factors (measured and unmeasured) are similar
Design Configuration Parallel group: random assignment to A vs B A Population B Randomization
Design Configuration Crossover: random assignment to AB or BA subject serves as own control A B Population B A Randomization Wash out period
Design Configuration Factorial Designs: interest in comparing A vs B and C vs D randomly assign to A+C, A+D, B+C or B+D C A D Population B C Randomization D
[Other Study Designs] Cohort Studies Case-Control Studies Descriptive Studies (e.g., surveys)
Type of Comparison Trials to show superiority new treatment vs placebo, or new treatment vs active control test (and hopefully reject) the null hypothesis that there is no difference in outcomes between groups vs. the alternative hypothesis that there is a difference between the groups one-sided or two-sided
Type of Comparison Trials to show equivalence new treatment vs. active control test (and hopefully reject) the null hypothesis that the new treatment performs differently than the active control by at least some small amount, vs. the alternative hypothesis that the difference between the groups is no greater than this small amount an amount that is sufficiently small that the treatments are considered equivalent for all practical purposes if the difference between the treatments is smaller than this amount (equivalence margin)
Type of Comparison Trials to show noninferiority new treatment vs. active control test (and hopefully reject) the null hypothesis that the active control performs better than the new treatment by at least some small amount vs. the alternative hypothesis that the new treatment does not perform worse than the active control by more than this small amount
Type of comparison Trials to show equivalence e.g., test null hypothesis that % cured with active control is 5% more than the % cured with the new treatment and vice versa Trials to show noninferiority e.g., test null hypothesis that % cured with active control is 5% than the % cured with the new treatment
Sample Size Determine based on: a primary endpoint the null hypothesis the test statistic (e.g., t-test, chi-square test, logrank test) the treatment difference to be detected (the alternative hypothesis ) significant level (type I error) desired power (type II error) variability assumptions the plan for handling treatment withdrawals and protocol violations
Data Monitoring Oversight of Trial Quality Monitoring of Treatment Effects
Oversight of Trial Quality Checks perform in a blinded manner: whether the protocol is being followed the acceptability of data being accrued the success of planned accrual targets the appropriateness of the design assumptions success in keeping patients in the trials Has no impact on type I error
Monitoring Treatment Effects: Interim Analysis Usually for serious outcomes Requires unblinded access to treatment group summary data Should only be done if included in the protocol Goal, stop the trial early if: superiority of the new treatment is clear future demonstration of a treatment effect is unlikely unacceptable adverse effects are apparent
Monitoring Treatment Effects: Interim Analysis May requiere a IDMC, which should approve interim plans Study Team PI Statistical Center IDMC Sponsor IRB
Monitoring Treatment Effects: Interim Analysis Repeated testing of outcome data increases the chance of a type I error Test for difference in proportions failing in two groups without adjustment for multiple testing Decision Rule: Reject null if Z 1.96 Overall Type I Error Rate Single test at end of study -----> 0.05 Two tests, equally spaced -----> 0.08 Five tests, equally spaced -----> 0.14 (Friedman, Furberg and DeMets, 1996)
Data Analysis Include main features of analysis in protocol For confirmatory trial, include statistical methods to be used for the primary variable(s) For exploratory trials, include general principles and directions Additional statistical analysis plan detailed procedures for primary and secondary variables do blind review of data, record date of breaking blind
Analysis Sets Disposition of participants enrolled, summary of protocol violations Degree of compliance and missing data lead to different Analysis Sets: Full Analysis Set Per Protocol Set Rationale: Minimize bias (Analysis Sets defined a priori) Demonstrate lack of sensitivity
Full Analysis Set Full analysis set = the analysis set which is as complete as possible and as close as possible to the intention-to-treat ideal of including all randomized subjects, it may exclude, for example: Participants who failed to meet a major entry criteria Participants who lack any data post randomization
Full Analysis Set If specified in the plan, subjects who fail to meet an entry criterion may be excluded without the possibility of introducing bias under the following circumstances: the entry criterion was measured prior to randomization the detection of the relevant eligibility violations can be made completely objectively all subjects receive equal scrutiny for eligibility violations all detected violations of the particular entry criterion are excluded
Per Protocol Set Per protocol set = subset of the participants in the full analysis set who are more compliant with the protocol complete a certain pre-specified minimal exposure to the treatment regimen have some minimum number of measurements of the primary variable(s) have no major protocol violations May give overly optimistic results in superiority trials May be the more conservative analysis set for equivalence or non-inferiority trials
Estimation and Confidence Intervals Not just p-values, include confidence intervals for estimated treatment effects Prespecify any covariates to be controlled for in primary or secondary analysis To improve precision To adjust for potential imbalances To account for stratified designs Never adjust for post-randomization variables Prespecify interactions and subgroups of interest if treatment effect is likely to vary by baseline factors (e.g., gender)
Evaluation of Safety Choice of variables: Laboratory tests, vital signs, adverse events Safety Analysis Set: Usually those who received at least one dose of the investigational drug Statistical Analysis
Reporting Document deviations from analysis plan, when and why they occurred Account for all subjects who entered the study Describe all reasons for exclusion from analysis dataset and all protocol violations Summarize measurements of all important variables Consider the effect of loss of subjects, violations and missing data on analysis results Describe participants lost, withdrawn, etc.