Analysis and Interpretation of Clinical Trials. How to conclude?

Size: px

Start display at page:

Download "Analysis and Interpretation of Clinical Trials. How to conclude?"

Toby Pearson
7 years ago
Views:

www.eurordis.org Analysis and Interpretation of Clinical Trials How to conclude?

1 Analysis and Interpretation of Clinical Trials How to conclude? Statistical Issues Dr Ferran Torres Unitat de Suport en Estadística i Metodología - USEM Statistics and Methodology Support Unit Hospital Clínic Barcelona Ferran.Torres@uab.es Barcelona, June 15, 2009

2 Documentation All the information will be available at the following link: Presentation in power point and pdf Additional documentation: Scientific recommendations Regulatory recommendations 2

name/docencia/eurordis Presentation in power point and pdf

3 Documentation 3

4 Outline Why statistics? Population and samples p value Statistical vs clinical significance Statistical errors (a and b) Sample size Estimation of treatment effect: Confidence Intervals 4

5 Why Statistics? Variation!!!! 5

6 Variability 6

7 Why Statistics? Medicine is a quantitative science but not exact Not like physics or chemistry Variation characterises much of medicine Statistics is about handling and quantifying variation and uncertainty Humans differ in response to exposure to adverse effects Example: not every smoker dies of lung cancer some non-smokers die of lung cancer Humans differ in response to treatment Example: penicillin does not cure all infections Humans differ in disease symptoms Example: Sometimes cough and sometimes wheeze are presenting features for asthma 7

about handling and quantifying variation and uncertainty Humans differ in response to exposure to adverse effects Example: not every

8 Why Statistics Are Necessary Statistics can tell us whether events could have happened by chance and to make decisions We need to use Statistics because of variability in our data Generalise: can what we know help to predict what will happen in new and different situations? 8

Statistics because of variability in our data Generalise: can what

9 Memorable quotes 50% of what you learn about therapy in the next 5 years is wrong. (The trouble is we don t know which 50%) (Anon) in this world there is nothing certain but death and taxes. Benjamin Franklin ( ). (also said by Woody Allen) 86% of all statistics are invented on the spot (Huff How to Lie with Statistics) There are lies, damn lies, and statistics Benjamin Disraeli ( ) 9

10 Error? unreliable reliable but not valid reliable & valid Classification of measurements 10

11 Random versus Systematic error Example: Systolic Blood Pressure (mm Hg) Random Systematic (Bias) True Value True Value

Systematic (Bias) True Value 130 150 170 01

12 Random versus Systematic error Random Sample size Bias Sample size 12

13 Types of Statistics Descriptive Statistics enumerate, organise, summarise, and categorise graphical representation of data. these type of statistics describes the data. Examples means and frequency of outcomes charts and graphs 13

14 Types of Statistics Inferential Statistics drawing conclusions from incomplete information. they make predictions about a larger population given a smaller sample these are thought of as the statistical test Examples 95%CI, t-test, chi-square test, ANOVA, regression 14

they make predictions about a larger population given a smaller

15 Population and Samples Sample Population of the Study Target Population 15

16 Extrapolation Sample Study Results Inferential analysis Statistical Tests Confidence Intervals Population Conclusions 16

17 Statistical Inference Statistical Tests=> p-value Confidence Intervals 17 17

18 Valid samples? Population Likely to occur Invalid Sample and Conclusions Unlikely to occur 18

19 P-value The p-value is a tool to answer the question: Could the observed results have occurred by chance*? p <.05 statistically significant Remember: Decision given the observed results in a SAMPLE Extrapolating results to POPULATION *: accounts exclusively for the random error, not bias

05 statistically significant Remember: Decision given the observed

20 RCT from a statistical point of view Treatment A Randomisation Treatment B (control) 1 homogeneous population 2 distinct populations 20 TITRE DE LA PRESENTATION (A MODIFIER SUR LE MASQUE)

21 RCT Sample Population 21

22 Statistical significance/confidence A>B p<0.05 means:? I can conclude that the higher values observed with treatment A vs treatment B are linked to the treatment rather to chance, with a risk of error of less than 5% 22 TITRE DE LA PRESENTATION (A MODIFIER SUR LE MASQUE)

23 Factors influencing statistical significance Signal Noise (background) Quantity Difference Variance (SD) Quantity of data 23 TITRE DE LA PRESENTATION (A MODIFIER SUR LE MASQUE)

24 P-value A very low p-value does NOT imply: Clinical relevance (NO!!!) Magnitude of the treatment effect (NO!!) With n or variability p Please never compare p-values!! (NO!!!) 24

25 Type I & II Error & Power Reality (Population) A=B A B Conclusion (sample) A=B p>0.05 A B p<0.05 OK Type I error (a) Type II error (b) OK 25

26 Type I & II Error & Power Type I Error (a) False positive Rejecting the null hypothesis when in fact it is true Standard: a=0.05 In words, chance of finding statistical significance when in fact there truly was no effect Type II Error (b) False negative Accepting the null hypothesis when in fact alternative is true Standard: b=0.20 or 0.10 In words, chance of not finding statistical significance when in fact there was an effect 26

27 Sample Size The planned number of participants is calculated on the basis of: Expected effect of treatment(s) Variability of the chosen endpoint Accepted risks in conclusion effect number variability number risk number 27

28 Frecuencia Frecuencia Frecuencia Sample Size The planned number of participants is calculated on the basis of: Expected effect of treatment(s) Variability of the chosen endpoint Accepted risks in conclusion effect number variability number risk number 300 ALTURA 300 ALTURA 120 ALTURA Desv. típ. = Media = N = Desv. típ. = Media = N = Desv. típ. = Media = N = ALTURA ALTURA ALTURA 28

29 Sample Size The planned number of participants is calculated on the basis of: Expected effect of treatment(s) Variability of the chosen endpoint Accepted risks in conclusion effect number variability number risk number A=B Reality (Population) A B Conclusion (sample) A=B p>0.05 A B p<0.05 OK Type I error (a) Type II error (b) POWER 29

30 95%CI Better than p-values use the data collected in the trial to give an estimate of the treatment effect size, together with a measure of how certain we are of our estimate CI is a range of values within which the true treatment effect is believed to be found, with a given level of confidence. 95% CI is a range of values within which the true treatment effect will lie 95% of the time Generally, 95% CI is calculated as Sample Estimate ± 1.96 x Standard Error 30

31 Interval Estimation A probability that the population parameter falls somewhere within the interval Confidence interval Sample statistic (point estimate) Confidence limit (lower) Confidence limit (upper) 31

32 Superiority study Control better Test better IC95% d < 0 - effect d = 0 No differences d > 0 + effect 32

33 Lower equivalence boundary Upper equivalence boundary Statistically and Clinically superiority Statistical Superiority Non-inferiority Equivalence Inferiority <- Treatment less effective 0 Treatment-Control Treatment more effective -> 33

34 34

II. DISTRIBUTIONS distribution normal distribution. standard scores

II. DISTRIBUTIONS distribution normal distribution. standard scores Appendix D Basic Measurement And Statistics The following information was developed by Steven Rothke, PhD, Department of Psychology, Rehabilitation Institute of Chicago (RIC) and expanded by Mary F. Schmidt,