BMJ Open. When are clinical trials beneficial for both study patients and future patients?

Transcription

1 When are clinical trials beneficial for both study patients and future patients? Journal: BMJ Open Manuscript ID bmjopen-0-00 Article Type: Research Date Submitted by the Author: -Jan-0 Complete List of Authors: Mhaskar, Rahul; University of South Florida College of Medicine, Internal Medicine, Program for Comparative Effectiveness Research Evidence-based Medicine Miladinovic, Branko; University of South Florida College of Medicine, Internal Medicine, Program for Comparative Effectiveness Research Evidence-based Medicine Gutterbock, Thomas; University of Virginia, Center for Survey Research Djulbegovic, Benjamin; University of South Florida, Internal Medicine, Program for Comparative Effectiveness Research Evidence-based Medicine <b>primary Subject Heading</b>: Ethics Secondary Subject Heading: Research methods Keywords: Uncertainty, Institutional review board, Approval, Vignette, Factorial Survey, Pragmatic trials -

2 Page of 0 BMJ Open Title: When are clinical trials beneficial for both study patients and future patients? Authors: Rahul Mhaskar, PhD, MPH, Branko Miladinovic, PhD, Thomas M. Guterbock, PhD, Benjamin Djulbegovic, MD, PhD Author affiliations: USF Program for Comparative Effectiveness Research and Evidence-Based Medicine, Morsani College of Medicine, University of South Florida University of Virginia, Center for Survey Research Corresponding author: Rahul Mhaskar, PhD, MPH Assistant Professor USF Program for Comparative Effectiveness Research, Evidence Based Medicine Department of Internal Medicine, Morsani College of Medicine, University of South Florida 0 Bruce B Downs Blvd, MDC, Tampa, FL, USA Phone: --0 Fax: rmhaskar@health.usf.edu Key words: Uncertainty, Institutional review board, Approval, Vignette, Factorial Survey, Pragmatic trials Word count: Text:, -

3 Page of Abstract Objective: The ethicists believe that the goal of clinical research is to benefit future and not current (trial) patients. Many clinicians believe that the clinical trial enrollment offers best management for their patients. Objective of our study was to identify the situations when a clinical trial is beneficial for both the patients enrolled in the trial and future patients. Design: factorial vignette based cross-sectional survey via the internet Participants: Institutional Review Board (IRB) members of the US Medical Schools Main outcome measures: Each participant was invited to review nine clinical vignettes related to a) study approval, b) the assessment if the study is designed to help future or current patients more. Results: A total of IRB members from institutions participated. When considering approval of the trial, we found that uncertainty about treatment effects [OR=. (.0,.)] and requirement for continuation of standard therapy [OR=. (.-.)] were only statistically significant factors affecting IRB members decisions to approve the study. The odds of IRB members approving a trial increased when a trial proposed to enroll patients with lifethreatening versus chronic debilitating disease (OR =.0, %CI:.-.). We also found that similar factors affected judgments related to the assessment whether the trial will benefit future or current patients more: A) Future patients: uncertainty [OR=.(.,.)]; continuation of standard treatment [OR=.(.0-.]; seriousness of condition [OR=. -

4 Page of 0 BMJ Open (.-.)]; B) current patients: uncertainty [OR=.(.,.)]; continuation of standard therapy [.(.-.)]. Conclusions: IRB members view the proposed studies to be beneficial for both current patients and future patients if there is uncertainty regarding treatment effects and if studies do not require stopping the current treatment. This finding supports the design and use of pragmatic trials which may reduce therapeutic misconception and improve trial enrollment speeding up therapeutic discoveries. -

5 Page of Strengths and limitations of this study: Strengths: This is the first study not only exploring the IRB member s decision making process regarding approval of a research study but also the factors that they consider most important for approval of the proposed study because it is deemed beneficial for both study and future patients. We conducted a study to find the common ground between these two diametrically opposite positions by identifying those clinical situations when enrollment into clinical trials serves equally well both trial and future patients. We found that IRB members view the proposed studies to be beneficial for both current patients and future patients if there is uncertainty regarding treatment effects and if studies do not require stopping the current treatment. Limitation: A limitation of our study is that our target population did not include the members of commercial IRBs, and there is no published research on the issue of what factors impact their decisions of approving trials. -

6 Page of 0 BMJ Open Introduction Key ethical research documents such as Belmont report and Declaration of Helsinki state that clinical research is done with the purpose of benefiting future patients and not study patients. However, health care professionals are duty-bound not to subjugate their duties to patients best interest to the utilitarian goals for the good of others. From this perspective, enrollment into clinical studies is justified only if it benefits study patients more than those outside of the trials. As a result, many clinicians believe that enrolling patients in well-designed studies benefit patients more than treating them outside of the research protocols. For example, the influential National Cancer Center Network (NCCN) states that NCCN believes that the best management of any cancer patient is in a clinical trial. This disagreement about the nature of clinical research among professionals has led to therapeutic misconception (TM) among study participants. Therapeutic misconception is a state that exists when individuals do not understand that the defining purpose of clinical research is to produce generalizable knowledge regardless of whether the subjects enrolled in the trial may potentially benefit from the intervention under study. However, clinical research and clinical practice are not mutually exclusive categories and the mixing of boundaries between research and practice further fuels TM. Researchers have identified key factors related to blurring of boundaries between treatment and research leading to TM. Which of these factors influences decisions regarding categorizing a proposed research as helping patients in the trial proposed versus future patients is, however, not known. We hypothesize that it is possible to identify the situations where the interests of trial patients and -

7 Page of future patients are aligned. In turn, this can help avoid TM and provide better ethical and scientific framework to foster faster and more reliable medical discoveries. Methods: The target sample for this study consisted of active university IRB members in the United States and was identified based on multiple strategies member s lists from the Association of American Medical Colleges and the Public Responsibility in Medicine and Research. We conducted a factorial vignette based cross-sectional web based survey in which clinical uncertainties and other factors were incorporated into several hypothetical clinical research studies. We used a factorial design whereby seven aspects of each scenario were randomly varied in phrases (etable ) to produce unique vignettes. Each respondent was presented with four pairs of vignettes, with each pair representing one of the four types of clinical studies [phase I, phase II, randomized controlled trial (RCT), and a cohort study] and one randomly selected vignette. At the end of each vignette the participants responded to the following three questions based on a -point Likert-type scale: () will proposed study generate knowledge about medical treatment benefitting future patients (i.e., those not enrolled in the proposed trial); () will the proposed study help improve outcomes in the patients enrolled in the trial ( current patients); and () will you approve the proposed study? Participants were asked to assume that the described study was scientifically sound and appropriate. Participants received a $ gift card sent with advance letters. The study was approved by the University of South Florida IRB (No: 0). Statistical analyses -

8 Page of 0 BMJ Open We evaluated the impact of the seven factors (etable ) on IRB members approval of trial and their perceived benefit for current and future patients using a multilevel mixed-effects logistic regression with QR decomposition. The proportion of the total variation in outcome explained by clustering is expressed using the intra-class correlation coefficient. All continuous variables were centered on the mean. Subgroup analyses according to study design (for phase I to III, phase III RCT only and observational study only) were performed. Summary data are reported as odds ratio (OR) along with % confidence intervals (CI). All analyses were performed using the Stata (version.) software. Results: A total of IRB members from institutions with at least one person completing the survey participated in our study (Table ). Factors associated with approval of a trial When considering approval of the trial, we found that uncertainty about treatment effect (OR=.; %CI:.0,.) and no requirement for discontinuing standard therapy (OR=.; %CI:.-.) were only statistically significant factors associated with IRB members decisions to approve the study. In addition, the odds of IRB members approving a trial increased when a trial proposed to enroll patients with life-threatening vs. chronic debilitating disease [pancreatic cancer rather than rheumatoid arthritis (OR =.0; %CI:.-.)]. These findings were consistent across combined sets of phase I, II and III studies vs. phase III RCT only. For observational studies alone no requirement for stopping standard therapy (OR = -

9 Page of ; % CI:.-0) and condition/disease severity (OR =.; % CI:.-0) were associated with trial approval. Factors associated with benefit of current patients Uncertainty about treatment effect (OR =.; %CI:.-.) and no requirement for stoppage of standard therapy i.e., continuation of the existing treatment (OR =.; % CI:.-.) were only statistically associated factors affecting IRB judgment whether the proposed trials would benefit current patients (Table ). These findings were derived from the analysis of combined sets of phase I, II and III studies. For Phase III RCTs only and observational studies alone, none of the variables were statistically significantly associated with IRB members judgments whether the proposed trial would benefit current patients. Factors associated with benefit of future patients Uncertainty about treatment effects (OR =.; %CI:.-.), condition/disease severity (OR =.; %CI:.-.) and no requirement for discontinuing standard therapy (OR =.; % CI:.0-.) were statistically associated with trial benefitting future patients (Table ). These findings were emerged from the analysis of combined sets of phase I, II and III studies. For Phase III RCTs only none of the variables were statistically significantly associated with members suggesting that the proposed trial would benefit current patients. For observational studies alone, condition/disease severity was the only factor that affected IRB members judgments whether the proposed study would benefit future patients (OR =.; % CI:.-). -

10 Page of 0 BMJ Open None of the baseline demographic factors (Table ) were significantly associated with the IRB members approval of the trial, or whether they judged the study to benefit future or current patients. Discussion: This is the first study not only exploring the IRB member s decision making process regarding approval of a research study but also the factors that they consider most important for approval of the proposed study because it is deemed beneficial for both study and future patients. We found that IRB members consider two factors (uncertainty about treatment effects and continuation of standard therapy) most important for approval of the proposed trial, which, if adhered to, can benefit both future and study patients. The findings that uncertainty about treatment effect is important consideration for the trial approval is in line with the established normative precepts that research is conducted to address existing uncertainties (Figure ). Our study also showed that IRB members are % more likely to approve a trial which allows the enrolled participants to continue use of standard therapy compared with a trial with the strict protocol adherence requiring discontinuation of the standard treatment. Thus, IRB members i.e., people who are charged with oversight of clinical research appear to support the design and use of pragmatic trials as the ethically and scientifically most robust research design. According to our findings, pragmatic trials identify the situations where the interests of trial patients and future patients are aligned; they help avoid the problem of therapeutic misconception and, in turn, speed up therapeutic discoveries. The foundation of a pragmatic trial is the capability to appraise an intervention's effectiveness -

11 Page 0 of in real life and achieve maximum external validity, i.e., to generalize results to many settings -. Our findings provide additional ethical support to comparative effectiveness research, which typically relies on the use of pragmatic trials.. It is expected from our ethical analysis that use of pragmatic trials will bridge gaps in understanding between clinicians and researchers and ultimately lead to improved patient outcomes. Conclusions We conclude that no requirement for discontinuing of standard therapy and considerations of the existing uncertainties about treatment effects are the key design attributes that should be taken into account when proposing new trials. We believe that before approving the trials, IRBs should introduce the policy to better assess the existing uncertainties in a given trial employing methods such as survey of experts, publication of the protocols, and systematic reviews with a focus on identifying best standard therapy that should be retained in consideration of the proposed trial. Our ethical analysis supports the use of pragmatic trials as the mechanism to reconcile the differences between primary intentions of physicians (to make the best treatment available to their patients) and researchers (to test new treatments under ideal conditions to benefit future patients). This, we believe, will reduce ethical misgivings about trial enrollment potentially improving enrollment, a key pre-requisite to accelerate the discovery process and help improve patient outcomes

12 Page of 0 BMJ Open Contributions and sources: This study was funded by a grant from the Florida Department of Health under the Bankhead-Coley Cancer Research Program (grant# 0BW-0, PI Dr. Djulbegovic, B.). The sponsor had no role in design, data collection, analysis and reporting (manuscript design). RM designed the survey, assisted in data collection, wrote the first draft. BM conducted the data analyses. TG assisted in study implementation, participant recruitment, assisted in data collection. BD designed the study, designed the survey, assisted in data collection and supervised the entire project. The authors would like to thank the University of Virginia Survey Research Center faculty and staff who worked on the study, particularly the late Robin Bebel, Andy Lin, Huili Tang and Dave Shreve. Finally, the authors would like to thank the participants who donated their valuable time to assist with this research study. Competing interests disclosure statement: We have read and understood BMJ policy on declaration of interests and declare that we have no competing interests. Data sharing statement: The statistical code and dataset will be made available upon request. -

13 Page of References:. Foster C. The ethics of medical research on humans. Cambridge: Cambridge University Press, 00.. Djulbegovic B. Articulating and responding to uncertainties in clinical research. J Med Philosophy 00;:-.. National Comprehensive Cancer Network. National Comprehensive Cancer Network. Fort Washington, PA, 0.. Henderson GE, Churchill LR, Davis AM, Easter MM, Grady C, Joffe S, et al. Clinical trials and medical care: defining the therapeutic misconception. PLoS medicine 00;():e.. Peppercorn JM, Weeks JC, Cooke EF, Joffe S. Comparison of outcomes in cancer patients treated within and outside clinical trials: conceptual framework and structured review. Lancet 00; -0.. Vist GE, Hagen KB, Devereaux PJ, Bryant D, Kristoffersen DT, Oxman AD. Systematic review to determine whether participation in a trial influences outcome. BMJ 00;0(0):-.. STATA [program]. College Station, Texas, 0.. Djulbegovic B. Uncertainty and Equipoise: At Interplay Between Epistemology, Decision-Making and Ethics. The American journal of the medical sciences 0;():-.. Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials 00;0:. 0. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. Journal of chronic diseases ;0():-.. Ofman JJ, Lubeck DP. Realizing the benefits of practical clinical trials. JAMA : the journal of the American Medical Association 00;():-; author reply.. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA : the journal of the American Medical Association 00;0():-.. Maclure M. Explaining pragmatic trials to pragmatic policymakers. Journal of clinical epidemiology 00;():-.. Mann H, Djulbegovic B. Choosing a control intervention for a randomised clinical trial. BMC medical research methodology 00;:. -

14 Page of 0 BMJ Open Figure legends Figure. Box plot of the probability of approving the trial as a function of uncertainty for all study designs combined. -

15 Page of Table. Participant demographics Baseline Variables Frequency (%) or Median (Range) Gender (n = ) Male (.) Female (.) Age (n = ) (0-0) Years on IRB (n=) (-) Academic rank (n=) Instructor (.) Assistant Professor/Member (.) Associate Professor/Member () Professor/Senior Member (.) Professor Emeritus (.) NA/Other (.) Highest level of education (n = ) Bachelors/Masters a (.) Doctorate (PhD, PharmD, EdD) (.) Medical Doctor (MD) b (.) Consider yourself as scientist (n = ) Yes 0 (0.) No (.) Type of Review Board (n = ) Behavioral studies (.) Medical studies 0 (.) Role in IRB* Board Chair Committee Chair Research compliance Reviewer Privacy officer/ Attorney Community representative Content expert Alternate member Other 0 a Includes one member with an Associate s degree. b Includes one member with a DDS. *Variable with overlapping categories. -

16 Page of 0 BMJ Open Table. Factors influencing approval of a trial and IRB members judgments whether the proposed trial will benefit current versus future patients Variable OR (% CI): trial approval OR (% CI): benefit of current patients OR (% CI): benefit of future patients Competing interest of a clinician as investigator vs. care-giver. (0.,.).0 (0.,.). (0.,.) No requirement for discontinuing of standard therapy (adherence to standard treatment). (.,.)**. (.,.)**. (.0,.)* Testing and procedures that are intended to generate scientific knowledge vs. necessary for patient care.0 (0.,.) 0. (0.,.).0 (0.,.) Purpose of the trial (explanatory vs. pragmatic). (0.,.). (0.,.). (0.,.) Therapy type (gene therapy vs. chemotherapy vs. over counter medications). (0.,.).0 (0.,.).0 (0.,.) Severity of condition/disease (lifethreatening vs. non-lifethreatening).0 (.,.) **. (0.,.). (.,.)** Uncertainty about treatment effects (varied from % to 00%). (.0,.)**. (.,.)**. (.,.)** OR=odds ratio; CI=confidence intervals; For binary outcome: *Significant at 0.0; **Significant at 0.00; ORs are baseline adjusted for gender, age, review board status, education level, years on IRB, self-reporting as a scientist and quiz score. -

17 Page of Box plot of the probability of approving the trial as a function of uncertainty for all study designs combined. xmm (00 x 00 DPI) -

18 Page of 0 BMJ Open Instructions etable: Sample vignette Please Figure review the following vignette carefully and answer the three questions following the vignette. In your review of this vignette, please assume that the design and methods of the study are scientifically sound and appropriate, even though they are not described in detail. Do not focus on providing criticism of information that is lacking (i.e., sample size, inclusion criteria) [P]. Instead, consider the overall description of the study. Please remember that as an Institutional Review Board member, you have a responsibility to both participants in the study and people who may later be given the treatment being tested if it is found to be safe and effective. We refer to people who may get the treatment at a later time as future patients. Metastatic pancreatic cancer [P] is a deadly [P] condition. Most metastatic pancreatic cancer patients live for three to four months after diagnosis. Currently, death [P] can be prevented in half of those patients suffering from Metastatic pancreatic cancer by prescribing an existing treatment (Treatment Y). An alternative treatment [P] (Treatment X) has been developed during the last years to treat another disease. Patients who have taken Treatment X and patients who have taken Treatment Y have experienced minimal adverse events. Some health care providers have been using Treatment X off label to treat patients with metastatic pancreatic cancer. Treatment X is not approved by the Food and Drug Administration (FDA) for the treatment of metastatic pancreatic cancer. Doctors are uncertain which of these two treatments is the most effective and the least toxic. If either drug fails to treat a patient s metastatic pancreatic cancer the patient will most certainly die [P]. A group of investigators have submitted a proposal to your Institutional Review Board proposing a randomized controlled trial (RCT) testing whether treatment X is better than treatment Y in improving patients survival [P]. That is, their goal is to learn which of these two treatments improves survival [P] better under the ideal conditions of a research study and not necessarily [P] in a real medical practice. The proposed trial is a well-designed RCT in which treatment assignment is determined by chance (random assignment). In other words, there is a 0% [P0] chance of patients receiving Treatment X versus Treatment Y. The enrolled participants will be advised to follow the study treatment protocol, which will require stopping all concurrent metastatic pancreatic cancer treatment(s) upon initiation of the study. The RCT targets 0-0 year old adult patients. The results of a survey of 00 leading experts on metastatic pancreatic cancer are available to you. The survey indicates that 0 experts [P] favor treatment X and 0 experts [P] favor treatment Y. Participation in this trial does [P] requires additional testing and procedures over and beyond routine care. This might [P] include frequent blood tests, x-rays, and possibly endoscopy or other invasive procedures. The investigators want to enroll patients in this study because their primary goal is to advance scientific knowledge [P]. -

19 Page of etable: Factors that can affect trial approval Factors Levels * Uncertainty Total levels: expressed as probability levels ranging from 0% to 00% for Phase III RCT design; 0 and 00% for observational study; 0% and 0% for phase II study; % and % for phase I study) Severity of condition/disease Type of therapy Scientific purpose Study procedures Adherence to protocol Competing interest of a clinician as investigator vs. care-giver levels (life-threatening: pancreatic cancer vs. non-lifethreatening disorder: rheumatoid arthritis) levels (gene therapy vs. chemotherapy vs. over counter medications) levels (explanatory trials vs. pragmatic trials) levels (testing and procedures that are intended to generate scientific knowledge vs. necessary for patient care) levels (strict protocol-based schedule vs. no requirement for stoppage of standard therapy) levels (scientific focus vs. focus on patient care) *These are some of the key factors that IRB members likely take into considerations when approve a study. Almost infinitive number of levels can be conceived for many of these factors. However, for operational and testing purposes, we believe that the proposed characterization of factors/levels represent reasonable representation of the key domains that affect IRB members decision to approve or not approve a proposed study. -

20 Page of 0 BMJ Open STROBE Statement Checklist of items that should be included in reports of cross-sectional studies Item No Recommendation Title and abstract (a) Indicate the study s design with a commonly used term in the title or Introduction the abstract (b) Provide in the abstract an informative and balanced summary of what was done and what was found Background/rationale Explain the scientific background and rationale for the investigation being reported Objectives State specific objectives, including any prespecified hypotheses Methods Study design Present key elements of study design early in the paper Setting Describe the setting, locations, and relevant dates, including periods of recruitment, exposure, follow-up, and data collection Participants (a) Give the eligibility criteria, and the sources and methods of selection of participants Variables Clearly define all outcomes, exposures, predictors, potential Data sources/ measurement confounders, and effect modifiers. Give diagnostic criteria, if applicable * For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe comparability of assessment methods if there is more than one group Bias Describe any efforts to address potential sources of bias Study size 0 Explain how the study size was arrived at Quantitative variables Explain how quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen and why Statistical methods (a) Describe all statistical methods, including those used to control for Results confounding (b) Describe any methods used to examine subgroups and interactions (c) Explain how missing data were addressed (d) If applicable, describe analytical methods taking account of sampling strategy (e) Describe any sensitivity analyses Participants * (a) Report numbers of individuals at each stage of study eg numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up, and analysed (b) Give reasons for non-participation at each stage (c) Consider use of a flow diagram Descriptive data * (a) Give characteristics of study participants (eg demographic, clinical, social) and information on exposures and potential confounders (b) Indicate number of participants with missing data for each variable of interest Outcome data * Report numbers of outcome events or summary measures - Main results (a) Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (eg, % confidence interval). Make clear which confounders were adjusted for and why they were included - Page Number Table page -

21 Page 0 of (b) Report category boundaries when continuous variables were categorized (c) If relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period Other analyses Report other analyses done eg analyses of subgroups and interactions, and sensitivity analyses Discussion Key results Summarise key results with reference to study objectives Limitations Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both direction and magnitude of any potential bias Interpretation 0 Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence Generalisability Discuss the generalisability (external validity) of the study results Other information Funding Give the source of funding and the role of the funders for the present study and, if applicable, for the original study on which the present article is based *Give information separately for exposed and unexposed groups. Note: An Explanation and Elaboration article discusses each checklist item and gives methodological background and published examples of transparent reporting. The STROBE checklist is best used in conjunction with this article (freely available on the Web sites of PLoS Medicine at Annals of Internal Medicine at and Epidemiology at Information on the STROBE Initiative is available at -

22 When are clinical trials beneficial for both study patients and future patients? A factorial vignette based survey of institutional review board members Journal: BMJ Open Manuscript ID bmjopen-0-00.r Article Type: Research Date Submitted by the Author: -Apr-0 Complete List of Authors: Mhaskar, Rahul; University of South Florida College of Medicine, Internal Medicine, Program for Comparative Effectiveness Research Evidence-based Medicine Miladinovic, Branko; University of South Florida College of Medicine, Internal Medicine, Program for Comparative Effectiveness Research Evidence-based Medicine Gutterbock, Thomas; University of Virginia, Center for Survey Research Djulbegovic, Benjamin; University of South Florida, Internal Medicine, Program for Comparative Effectiveness Research Evidence-based Medicine <b>primary Subject Heading</b>: Ethics Secondary Subject Heading: Research methods Keywords: Uncertainty, Institutional review board, Approval, Vignette, Factorial Survey, Pragmatic trials -

23 Page of BMJ Open Title: When are clinical trials beneficial for both study patients and future patients? A factorial vignette based survey of institutional review board members Authors: Rahul Mhaskar, PhD, MPH, Branko Miladinovic, PhD, Thomas M. Guterbock, PhD, Benjamin Djulbegovic, MD, PhD Author affiliations: USF Program for Comparative Effectiveness Research and Evidence-Based Medicine, Morsani College of Medicine, University of South Florida University of Virginia, Center for Survey Research Corresponding author: Rahul Mhaskar, PhD, MPH Assistant Professor USF Program for Comparative Effectiveness Research, Evidence Based Medicine Department of Internal Medicine, Morsani College of Medicine, University of South Florida 0 Bruce B Downs Blvd, MDC, Tampa, FL, USA Phone: --0 Fax: rmhaskar@health.usf.edu Key words: Uncertainty, Institutional review board, Approval, Vignette, Factorial Survey, Pragmatic trials Word count: Text:, -

24 Page of Abstract Objective: The ethicists believe that the goal of clinical research is to benefit future and not current (trial) patients. Many clinicians believe that the clinical trial enrollment offers best management for their patients. Objective of our study was to identify the situations when a clinical trial is beneficial for both the patients enrolled in the trial and future patients. Design: factorial vignette based cross-sectional survey via the internet Participants: Institutional Review Board (IRB) members of the US Medical Schools Main outcome measures: Each participant was invited to review nine clinical vignettes related to a) study approval, b) the assessment if the study is designed to help future or current patients more. Results: A total of IRB members from institutions participated. When considering approval of the trial, we found that uncertainty about treatment effects [OR=. (.0,.)] and requirement for continuation of standard therapy [OR=. (.-.)] were only statistically significant factors affecting IRB members decisions to approve the study. The odds of IRB members approving a trial increased when a trial proposed to enroll patients with lifethreatening versus chronic debilitating disease (OR =.0, %CI:.-.). We also found that similar factors affected judgments related to the assessment whether the trial will benefit future or current patients more: A) Future patients: uncertainty [OR=.(.,.)]; continuation of standard treatment [OR=.(.0-.]; seriousness of condition [OR=. -

25 Page of BMJ Open (.-.)]; B) current patients: uncertainty [OR=.(.,.)]; continuation of standard therapy [.(.-.)]. Conclusions: IRB members view the proposed studies to be beneficial for both current patients and future patients if there is uncertainty regarding treatment effects and if studies do not require stopping the current treatment. This finding supports the design and use of pragmatic trials which may reduce therapeutic misconception and improve trial enrollment speeding up therapeutic discoveries. -

26 Page of Strengths and limitations of this study: Strengths: This is the first study not only exploring the IRB member s decision making process regarding approval of a research study but also the factors that they consider most important for approval of the proposed study because it is deemed beneficial for both study and future patients. We conducted a study to find the common ground between these two diametrically opposite positions by identifying those clinical situations when enrollment into clinical trials serves equally well both trial and future patients. We pilot tested the vignettes among IRB members and conducted this factorial survey over the web. Limitation: A limitation of our study is that our target population did not include the members of commercial IRBs. We did not investigate the impact of inclusion versus exclusion of vulnerable populations in trials on IRB members approval in the current study as the number of vignettes would have (due to the factorial design) become unmanageable. -

27 Page of BMJ Open Introduction Key ethical research documents such as Belmont report and Declaration of Helsinki state that clinical research is done with the purpose of benefiting future patients and not study patients. However, health care professionals are duty-bound not to subjugate their duties to patients best interest to the utilitarian goals for the good of others. From this perspective, enrollment into clinical studies is justified only if it benefits study patients more than those outside of the trials. As a result, many clinicians believe that enrolling patients in well-designed studies benefit patients more than treating them outside of the research protocols. For example, the influential National Cancer Center Network (NCCN) states that NCCN believes that the best management of any cancer patient is in a clinical trial. This disagreement about the nature of clinical research among professionals has led to therapeutic misconception (TM) among study participants. Therapeutic misconception is a state that exists when individuals do not understand that the defining purpose of clinical research is to produce generalizable knowledge regardless of whether the subjects enrolled in the trial may potentially benefit from the intervention under study. However, clinical research and clinical practice are not mutually exclusive categories and the mixing of boundaries between research and practice further fuels TM. Researchers have identified key factors related to blurring of boundaries between treatment and research leading to TM. Which of these factors influences decisions regarding categorizing a proposed research as helping patients in the trial proposed versus future patients is, however, not known. We hypothesize that it is possible to identify the situations where the interests of trial patients and -

28 Page of future patients are aligned. In turn, this can help avoid TM and provide better ethical and scientific framework to foster faster and more reliable medical discoveries. Methods: The target sample for this study consisted of active university IRB members in the United States and was identified based on multiple strategies member s lists from the Association of American Medical Colleges and the Public Responsibility in Medicine and Research. We conducted a factorial vignette based cross-sectional web based survey in which clinical uncertainties and other factors were incorporated into several hypothetical clinical research studies. We used a factorial design whereby seven aspects of each scenario were randomly varied in phrases (etable ) to produce unique vignettes. Each participant reviewed a total of vignettes. That is, each participant reviewed four pairs of vignettes, with each pair representing one of the four types of clinical studies [phase I, phase II, randomized controlled trial (RCT), and a cohort study] and one vignette randomly selected representing one of the four study designs. At the end of each vignette the participants responded to the following three questions based on a -point Likert-type scale: () will proposed study generate knowledge about medical treatment benefitting future patients (i.e., those not enrolled in the proposed trial); () will the proposed study help improve outcomes in the patients enrolled in the trial ( current patients); and () will you approve the proposed study? We pilot tested our vignettes (including language used for instructions) among IRB members before implementing the survey. Moreover, each vignette had the instructions clearly stated. It was specifically stated that In your review of this vignette, please assume that the design and -

29 Page of BMJ Open methods of the study are scientifically sound and appropriate, even though they are not described in detail. Do not focus on providing criticism of information that is lacking (i.e., sample size, inclusion criteria) (etable ). Participants received a $ gift card sent with advance letters. The study was approved by the University of South Florida IRB (No: 0). Statistical analyses We evaluated the impact of the seven factors (etable ) on IRB members approval of trial and their perceived benefit for current and future patients using a multilevel mixed-effects logistic regression with QR decomposition. The proportion of the total variation in outcome explained by clustering is expressed using the intra-class correlation coefficient. All continuous variables were centered on the mean. Subgroup analyses according to study design (for phase I to III, phase III RCT only and observational study only) were performed. Summary data are reported as odds ratio (OR) along with % confidence intervals (CI). All analyses were performed using the Stata (version.) software. Results: A total of IRB members from institutions with at least one person completing the survey participated in our study (Table ). Factors associated with approval of a trial When considering approval of the trial, we found that uncertainty about treatment effect (OR=.; %CI:.0,.) and no requirement for discontinuing standard therapy (OR=.; %CI:.-.) were only statistically significant factors associated with IRB members -

30 Page of decisions to approve the study. In addition, the odds of IRB members approving a trial increased when a trial proposed to enroll patients with life-threatening vs. chronic debilitating disease [pancreatic cancer rather than rheumatoid arthritis (OR =.0; %CI:.-.)]. These findings were consistent across combined sets of phase I, II and III studies vs. phase III RCT only. For observational studies alone no requirement for stopping standard therapy (OR =.; % CI:.-0) and condition/disease severity (OR =.; % CI:.-0) were associated with trial approval. Factors associated with benefit of current patients Uncertainty about treatment effect (OR =.; %CI:.-.) and no requirement for stoppage of standard therapy i.e., continuation of the existing treatment (OR =.; % CI:.-.) were only statistically associated factors affecting IRB judgment whether the proposed trials would benefit current patients (Table ). These findings were derived from the analysis of combined sets of phase I, II and III studies. For Phase III RCTs only and observational studies alone, none of the variables were statistically significantly associated with IRB members judgments whether the proposed trial would benefit current patients. Factors associated with benefit of future patients Uncertainty about treatment effects (OR =.; %CI:.-.), condition/disease severity (OR =.; %CI:.-.) and no requirement for discontinuing standard therapy (OR =.; % CI:.0-.) were statistically associated with trial benefitting future patients (Table ). These findings were emerged from the analysis of combined sets of phase I, II and III studies. For Phase III RCTs only none of the variables were statistically significantly associated with -

31 Page of BMJ Open members suggesting that the proposed trial would benefit current patients. For observational studies alone, condition/disease severity was the only factor that affected IRB members judgments whether the proposed study would benefit future patients (OR =.; % CI:.-). None of the baseline demographic factors (Table ) were significantly associated with the IRB members approval of the trial, or whether they judged the study to benefit future or current patients. Discussion: This is the first study not only exploring the IRB member s decision making process regarding approval of a research study but also the factors that they consider most important for approval of the proposed study because it is deemed beneficial for both study and future patients. We found that IRB members consider two factors (uncertainty about treatment effects and continuation of standard therapy) most important for approval of the proposed trial, which, if adhered to, can benefit both future and study patients. The findings that uncertainty about treatment effect is important consideration for the trial approval is in line with the established normative precepts that research is conducted to address existing uncertainties (Figure ). Our study also showed that IRB members are % more likely to approve a trial which allows the enrolled participants to continue use of standard therapy compared with a trial with the strict protocol adherence requiring discontinuation of the standard treatment. Thus, IRB members i.e., people who are charged with oversight of clinical research appear to support the design and use of pragmatic trials as the ethically and -

32 Page 0 of scientifically most robust research design. According to our findings, pragmatic trials identify the situations where the interests of trial patients and future patients are aligned; they help avoid the problem of therapeutic misconception and, in turn, speed up therapeutic discoveries. The foundation of a pragmatic trial is the capability to appraise an intervention's effectiveness in real life and achieve maximum external validity, i.e., to generalize results to many settings -. Our findings provide additional ethical support to comparative effectiveness research, which typically relies on the use of pragmatic trials. It is expected from our ethical analysis that use of pragmatic trials will bridge gaps in understanding between clinicians and researchers and ultimately lead to improved patient outcomes. In this study we focused only on key factors which IRB members may consider while approving the studies. Obviously there could be several other factors such as inclusion of vulnerable populations and informed consent process which might be considered by IRB members while approving a proposed study. We did not investigate the impact of inclusion versus exclusion of vulnerable populations in trials on IRB members approval in the current study as the number of vignettes would have (due to the factorial design) become unmanageable. Hence we have focused on the studies designs in which majority of the population are recruited and studied. Conclusions We conclude that no requirement for discontinuing of standard therapy and considerations of the existing uncertainties about treatment effects are the key design attributes that should be taken into account when proposing new trials. We believe that before approving the trials, IRBs should introduce the policy to better assess the existing uncertainties in a given trial employing methods such as survey of experts, publication of the protocols, and systematic reviews with a 0 -

33 Page of BMJ Open focus on identifying best standard therapy that should be retained in consideration of the proposed trial. Our ethical analysis supports the use of pragmatic trials as the mechanism to reconcile the differences between primary intentions of physicians (to make the best treatment available to their patients) and researchers (to test new treatments under ideal conditions to benefit future patients). This, we believe, will reduce ethical misgivings about trial enrollment potentially improving enrollment, a key pre-requisite to accelerate the discovery process and help improve patient outcomes. -

34 Page of Contributions and sources: This study was funded by a grant from the Florida Department of Health under the Bankhead-Coley Cancer Research Program (grant# 0BW-0, PI Dr. Djulbegovic, B.). The sponsor had no role in design, data collection, analysis and reporting (manuscript design). RM designed the survey, assisted in data collection, wrote the first draft. BM conducted the data analyses. TG assisted in study implementation, participant recruitment, assisted in data collection. BD designed the study, designed the survey, assisted in data collection and supervised the entire project. The authors would like to thank the University of Virginia Survey Research Center faculty and staff who worked on the study, particularly the late Robin Bebel, Andy Lin, Huili Tang and Dave Shreve. Finally, the authors would like to thank the participants who donated their valuable time to assist with this research study. Competing interests disclosure statement: We have read and understood BMJ policy on declaration of interests and declare that we have no competing interests. Data sharing statement: The statistical code and dataset will be made available upon request. -

35 Page of BMJ Open References:. Foster C. The ethics of medical research on humans. Cambridge: Cambridge University Press, 00.. Djulbegovic B. Articulating and responding to uncertainties in clinical research. J Med Philosophy 00;:-.. National Comprehensive Cancer Network. National Comprehensive Cancer Network. Fort Washington, PA, 0.. Henderson GE, Churchill LR, Davis AM, Easter MM, Grady C, Joffe S, et al. Clinical trials and medical care: defining the therapeutic misconception. PLoS medicine 00;():e.. Peppercorn JM, Weeks JC, Cooke EF, Joffe S. Comparison of outcomes in cancer patients treated within and outside clinical trials: conceptual framework and structured review. Lancet 00; -0.. Vist GE, Hagen KB, Devereaux PJ, Bryant D, Kristoffersen DT, Oxman AD. Systematic review to determine whether participation in a trial influences outcome. BMJ 00;0(0):-.. STATA [program]. College Station, Texas, 0.. Djulbegovic B. Uncertainty and Equipoise: At Interplay Between Epistemology, Decision-Making and Ethics. The American journal of the medical sciences 0;():-.. Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials 00;0:. 0. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. Journal of chronic diseases ;0():-.. Ofman JJ, Lubeck DP. Realizing the benefits of practical clinical trials. JAMA : the journal of the American Medical Association 00;():-; author reply.. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA : the journal of the American Medical Association 00;0():-.. Maclure M. Explaining pragmatic trials to pragmatic policymakers. Journal of clinical epidemiology 00;():-.. Mann H, Djulbegovic B. Choosing a control intervention for a randomised clinical trial. BMC medical research methodology 00;:. -