1 Study Design for Chemoprevention Cancer Epidemiology, Prevention and Control Workshop Shanghai, March 12, 2008
3 I. INTRODUCTION Experimental studies are conducted to assess the effect of a treatment using a drug, or intervention using a preventive agent, etc. Treatment Prevention Early detection/screening Diagnostic Quality of life/supportive care
4 A. Defined: A study design in which the investigator actively controls who is exposed and who is not. Subjects are randomly assigned to various treatment groups and followed to observe outcomes.
6 Comparison of Two Study Designs
8 Stratified Randomization
9 B. Experimental studies compared to cohort studies 1. Similarities between cohort and experimental studies. Both: a. Subjects must be free of the outcome at the start of the study. b. People are grouped into exposed / not exposed categories. c. Groups are followed for a period of time to determine outcome. d. Yield incidence data so allow the calculation of risk and related measures. e. Susceptible to lost-to to-follow-up bias.
10 B. Experimental studies compared to cohort studies 2. Differences between cohort and experimental studies: a. Experimental studies involve active manipulation of exposure (treatment/alternative treatment), whereas in cohort studies, the investigator must merely observe the effect of exposure. b. Random allocation (or randomization) is an essential part of a good experimental study. Not possible in a cohort. c. Ethical issues often a major issue in experimental epidemiological studies. d. Compliance with study protocol is an important concern in experimental studies.
11 1. Defined: C. Random allocation/ randomization A procedure for assigning patients to experimental treatment and other treatment groups so that chance alone is responsible for the group assignment each subject has an equal chance of being in any of the treatment groups. 2. Purpose of randomization: To (attempt to) assure comparability of the study groups with respect to factors which may be related to outcome.
12 C. Random allocation/ randomization 3. IMPORTANT: Randomization is done after informed consent is obtained!!!
13 C. Random allocation/ randomization 4. Do Not Confuse Random Allocation with Random Selection!!! Random selection of subjects: A procedure for selecting subjects so that each has the same chance of being included in the study. When we can t afford to use all possible subjects in the source population. Purpose: To assure representativeness of subjects (of source pop.) When used? In any type of study design where a sample of the population is being selected.
15 C. Random allocation/ randomization 5. Randomization does not guarantee similarity of groups
16 D. Uses of experimental study design: 1. Evaluate benefits of an intervention: Therapeutic Preventive 2. Confirm etiologic relationship
17 II. TWO MAJOR TYPES OF EXPERIEMENTAL STUDIES: Clinical and community trials
18 A. Randomized clinical trial (RCT): 1. Defined (phase III): An experimental study where the effectiveness of the intervention is being tested on individuals. Phase I trials How does the agent affect the human body? What dosage is safe? Phase II trials Does the agent or intervention have an effect on the disease? Phase III trials Is the new agent or intervention (or new use of a treatment) better than the standard? Participants have an equal chance to be assigned to one of two or more groups
19 A. Randomized clinical trial (RCT): 2. Clinical trials are conducted for the treatment/prevention of both infectious diseases and chronic diseases: a. Infectious diseases: field trials (often refers to vaccine trials) b. Chronic diseases: e.g. Women s Health Initiative (WHI):
20 B.4. Recruit study Population:
21 Effectiveness of Intervention
22 From: Writing Group for the Women s Health Initiative Investigators. Risks and benefits of the estrogen plus progestin in healthy postmenopausal women. Principal results from the Women s Health Initiative Randomized Controlled Trial.JAMA.2002;288:
23 B. Community trial 1. Defined: An experimental study where the effectiveness of an intervention is tested on a community. 2. Example of a community trial: fluoridation of water.
24 Fluoridation of Water
26 B. Community trial 3. Problems of conducting community trials: a. Obtaining an appropriate control group: 1) Same community before and after intervention 2) A control community: similar to experimental community with respect to possible confounders. b. Other problems: 1) It is hard to get individual s informed concent 2) Intervention not at individuals level 3) Collaboration of communities
27 III. STEPS IN CONDUCTING RANDOMIZED CLINICAL TRIALS
28 Diagram of a design flow:
29 1. Specify Hypothesis B.1. Steps
30 B.2. Steps 2. Specify target and source of populations:
31 B.3 Define endpoints/possible side effects of intervention:
32 B.5. Obtain informed consent from those willing to participate: 1. Informed consent: An agreement (signed) that the subject understands the benefits and risks of the study. 2. Human Subjects Protection Committees: act as watchdogs review research applications the be sure they comply with issues pertaining to protection of human subjects. To receive research money from NIH (and other agencies), these committees must approve grand proposals. Require that all subjects must read and sign Informed Consent form.
34 B.6 Randomize to treatment 1. Experimental treatment. 2. Alternative treatment ( controls ): a. The current standard treatment: b. Placebo: 1) Defined: An inert substance prepared to look as similar as possible to the experimental treatment. 2) Placebo effect: Am improvement on health, symptoms, due to fact of being treated and not due to the treatment!
36 Three advantages of the randomized design: Randomization removes the potential of bias in the allocation of subjects to the intervention group or to the control group; Randomization tends to produce comparable groups; that is, the measured or unknown prognostic factors and other characteristics of the subjects at the time of randomization will be, on the average, evenly balanced between the intervention and control groups; The internal validity of statistical tests of significance is guaranteed.
37 B.7. Blindness Fundamental Point: : To avoid potential problems of bias during data collection and assessment a clinical trial Ideally we should have a double-blind blind design. In studies where such a design is impossible, a single-blind approach and other measures to reduce potential bias are favored. Unblinded: : In an unblinded or open trial, both the subject and the investigator know to which intervention the subject has been assigned. The studies involving most surgical procedures, changes in life style (eating habits, exercise, smoking) or learning techniques can be conducted only in this manner. The advantage is that it is usually simpler to carry out and the disadvantage is the possibility of bias.
38 B.7. Blindness Single-Blind Blind: Patient does not know which treatment group he/she is in. Only the investigator is aware of which intervention each subject is receiving. The advantage and disadvantage are similar to those of unblinded trials. Double-Blind Blind: : Neither the subjects nor the investigators responsible for following the subjects know which intervention the subject has been assigned. Such designs are usually restricted to trials of drug efficacy. The advantage is that the risk of bias is reduced. The disadvantage is that certain responsibilities, which in open or single-blind studies could be accomplished by the investigators, must be taken over by others in order to maintain the blindness.
39 B.7. Blindness Triple-Blind Blind: : Neither the subjects, nor the investigators, nor the committee monitoring response variables is told the identity of the groups. The theoretical advantage is to allow the monitoring committee to evaluate the response variable results more objectively. The disadvantage is that in a trial where the monitoring committee has an ethical responsibility to ensure subject safety, such a design may be counterproductive.
41 B.7. Blinding (cont.) b. Purpose of blinding: To prevent biases in assessing outcome, which may be influenced by knowledge of treatment group. c. Blinding is most important when d. Blinding is less important when e. Not always possible to blind subjects and/or investigators
42 B.8 Follow-up study groups for outcomes
43 B.9. Analyze results: Use Intention to treat analyses 1. Defined: Data are analyzed so that subjects remain in groups as originally assigned even if subjects do not comply or change treatments on their own. This means that, even if the day after a subject was assigned to the control group, he starts the experimental treatment on his own the outcome for that patient is analyzed as if he were still in the control group!..in short: Once randomized, always analyzed.
44 B.9. Analyze results: Use Intention to treat analyses 2. Why we must use intention to treat analyses: a. The results reflect what happens in the real world when the treatment is offered. b. The trial concerns whether the offering of a new treatment is more effective. c. Non-compliers are often different with respect to outcome, so likely to be less bias in keeping them in original groups than analyzing data in other ways. d. Though final results must reflect intention-to to-treat treat analysis, investigators should look at compliers and non-compliers and their possible effect on the results.
45 B.10. Establish procedures for terminating the trial and informing subjects of results. To Protect the welfare of subjects, data must be monitored regularly and analyzed for any obvious benefits or clear risks to subjects. If either occurs, the trial must be stopped and appropriate action taken (if there is a benefit, then controls must be offered new treatment. If a risk, experimentals must be taken of exper. treatment.
46 B.10. Establish procedures for terminating the trial and informing subjects of results. Caveat: Small, transient differences between the groups may be observed early in the trial. So there must be a balance between a long-enough follow-up to see whether true benefits occur vs. depriving the controls of a better treatment.
47 IV. SOME MEASURES OF EFFECT IN EXPERIMENTAL STUDIES
48 A. Risk ratio: I / I C T
49 B. Risk difference: I C - I T
50 C. Efficacy: Ic It x 100 Ic
51 D. Example: 849 working adults between the ages of took part in a double-blind, blind, placebo-controlled, controlled, randomized trial of the effect of influenza vaccination on three outcomes: upper respiratory illness (URI), absenteeism from work due to URI, and visits to doctors for URI. Baseline characteristics were compared to assure comparability of the two groups. The subjects were followed from Dec. 1, 1994, to March 31, (Nichol, K.L. et al. N Engl J Med 1995; 333: ) 93.)
52 D. Examples a. The following tables shows the baseline characteristics of the two groups: does the randomization appear to have been successful, that is, do the two groups appear to be similar for those variables?
53 Baseline Assessment Fundamental Point: : Relevant baseline data should be measured in all study subjects before the start of the study. Baseline data include risk factors, prognostic factors, demographic and socioeconomic factors, and medical history. Baseline assessment can be used to assess comparability of the study groups. Although the randomization on the average produces balance between comparison groups, it does not guarantee balance in any specific trial. If baseline factors are not balanced, statistical adjustment methods such as stratified analysis and multivariate analysis can be used.
54 Baseline Assessment
55 D. Examples b. One concern in any experimental study, even if subjects are blinded, is whether subjects can tell whether they are in the treatment or control group. One way to determine is to ask the subject which group they think they are in. In the above study, 60% of the placebo and 54% of the vaccine recipients correctly identified their group (slightly better than chance along).
56 D. Examples c. Another way to look at this is to see if the two groups differ in reporting side effects. Table 2 shows the side effects reported by each of the groups. Does it look as if any of the side effects night give away a subject s status?
58 D. Example: d. Table 3 summarizes the data regarding the three outcomes, using rates per 100 subjects. For Episodes of UPI, calculate the three measures of effect: Risk ratio: Risk difference Efficacy (or Vaccine Effectiveness )
59 D. Example:
60 From: Writing Group for the Women s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women. Principal results from the Women s Health Initiative Randomized Controlled Trial.JAMA.2002;288:
61 Major potential biases in Lost-to to-follow-up experimental studies: Compliance/adherence to study s protocol, e.g., WHI:
62 Reasons for non-compliance: 1. Misunderstanding of instructions. 2. Inconvenience of participation. 3. Side effects of treatment. 4. Cost of participation. 5. Forgetfulness. 6. Disappointment with results. 7. Preference for another treatment.
63 Ways to improve compliance: 1. Select motivated persons. 2. Pretest ability and willingness of participants to comply. 3. Provide simple and lucid instructions to subjects. 4. Offer incentives to comply (e.g., no charge for therapeutic intervention and associated examinations). 5. Provide positive reinforcements to subjects for adherence to treatment regimen. 6. Maintain frequent contract with participants and remind them about importance of adherence to the regimen. 7. Measure adherence through pill counts or sampling of biologic specimens. 8. Limit duration of intervention.
64 Sample Size: Fundamental Point: : Clinical trials should have sufficient statistical power to detect differences between groups considered to be of clinical interest. Therefore, calculation of sample size with provision for adequate levels of significance and power is an essential part of planning. Clinical trials tend to be small. Even if there is a real difference in two treatments, the difference may not be statistically significant because of the small numbers. That is, the study did not have the power to detect (statistically) a real difference.
65 Sample size False positive (alpha-level, level, or Type I error). The alpha- level used and accepted traditionally are 0.01 or The smaller the level of alpha, the larger the sample size. False negative (beta-level, or Type II error). (1-beta) is called the power of the study. Investigator like to have a power of around 0.90 or 0.95 when planning a study, which means that there have a 90% or 95% chance of finding a statistically significant difference between study and control groups. The difference between study and control groups (delta). Two factors need to be considered here: one is what difference is clinically important, and the another is what is the difference reported by previous studies.
66 V. The Validity of Results of Epidemiologic Studies
67 A. Internal Validity 1. Defined: The truth or accuracy of results from a study with respect to its defined source population
68 2. Factors affecting internal validity: Information bias: distortion of true effects due to systematic errors in collection of data on exposure OR outcome Selection bias: distortion of true effects due to error in selection of subjects Confounding: distortion of true effects from other risk factors or exposures extraneous to the study
69 3. Algorithm for assessing flaws in an experimental study:
70 B. External Validity 1. Defined: Applicability of study results to other populations (ie( ie,, to specified target or reference population) Generalizability
71 2. Factors affecting external validity: a. Does the study have internal validity? external validity is not possible without internal (ie( ie,, a study must be internally valid first before it can also be externally valid) a. Is the study group representative of the target population? if the study group is non-representative, then the study is not externally valid
72 3. Algorithm for assessing external validity: Is experimental population representative of the reference population? YES No external validity problem You CAN extrapolate NO Would reference population have same response to treatment as experimental population? YES No external validity problem You CAN extrapolate NO External validity problem You CANNOT extrapolate