Analysis of numerical data S4

Transcription

1 Basic medical statistics for clinical and experimental research Analysis of numerical data S4 Katarzyna Jóźwiak 22nd November /44

2 Hypothesis tests: numerical and ordinal data 1 group: one-sample t-test, sign test. 2 groups: Dependent (paired/related): paired t-test, Wilcoxon signed rank test, sign test Independent (unrelated): unpaired t-test, Wilcoxon rank sum test. More than 2 groups: Dependent (paired/related): repeated measures ANOVA, Friedman s ANOVA Independent (unrelated): one-way ANOVA, Kruskal-Wallis test. 2/44

3 Single group One-sample t-test (parametric): H 0 : population mean = parameter value H 1 : population mean parameter value (or H 1 : population mean > parameter value, or H 1 : population mean < parameter value) Example: sample from a single group of individuals (high school students) with their standardized test scores in writing. H 0 : population mean writing score = 5 H 1 : population mean writing score 5 Reject H 0 if: sample mean-parameter value sample mean-parameter value < c or > c SEM SEM with c > 0. 3/44

4 Single group Variable (e.g., writing test score) is numerical, is normally distributed with a given (usually unknown) variance. If population variance unknown: test statistic follows a Student-t distribution: 5% significance level: SEM = s/ n and c = tn 1,1 α/2 = t n 1,0.975 (s =standard deviation of sample observations, n =sample size) If population variance known (σ 2 ) or sample size very large: test statistic follows a normal distribution (z-test): 5% significance level: SEM = σ/ n and c = z1 α/2 = z = /44

5 Two related groups Paired data (matched pairs): One group of individuals and variable measured on each individual in two circumstances. E.g. measurement while taking active treatment and while taking placebo; blood pressure measured before and after a particular treatment. Two samples of different individuals but linked to each other. E.g. matched patients in case-control study Example: sample of patients in which hours of sleep under a sleeping drug is measured one night, and hours of sleep under a placebo is measured a different night for each. 5/44

6 Two related groups Paired t-test (parametric): H 0 : population mean group 1 - population mean group 2 = parameter value For the hours of sleep example: H 0 : pop. mean hours sleep (drug) - pop. mean hours sleep (placebo) = 0 H 1 : pop. mean hours sleep (drug) - pop. mean hours sleep (placebo) 0 Since data are paired, we reduce the two samples to a single sample of differences: Difference in hours sleep = hours sleep under drug - hours sleep under placebo Variable (difference in hours sleep) is numerical, is normally distributed with a given (usually unknown) variance. Hypotheses become: H 0 : population mean difference hours sleep = 0 H 1 : population mean difference hours sleep 0 We can use a one-sample t-test. Ratio paired t-test: If ratio (treatment/control) seems better to quantify effect of treatment. 6/44

7 Two related groups Example SPSS output: Concentration of antibody (µg/ml) to type II group B Streptococcus in 20 volunteers before and after immunisation (Bland and Altman, 2009). H0 : population mean concentration before immunisation = population mean concentration after immunisation Paired Samples Test Paired Differences... Mean Std. Deviation Lower Pair Paired Samples Test Upper t df Sig. (2-tailed) Pair /44

8 Two related groups Wilcoxon signed ranks test (nonparametric): H 0 : population median of differences between paired observations = parameter value For the hours of sleep example: H 0 : population median difference hours sleep = 0 H 1 : population median difference hours sleep 0 Intuition: if the population median of differences is zero, then approximately half of the values of differences between the two samples should be below zero. No normality required, but it does assume (rough) symmetry. Robust to outliers Applicable to numerical and ordinal data 8/44

9 Two related groups Wilcoxon signed ranks test (nonparametric): How does it work? Compute differences between paired observations and ignore differences equal to the parameter value (sample size is reduced to n r ) Assign ranks to the absolute values of the differences (for ties, calculate the average rank) Reassign the + and - signs to the ranks Sum up positive ranks and negative ranks separately Test statistic W is the smaller value of the sum of positive ranks and sum of negative ranks Reject H0 if: Small sample size: W c with c > 0 Large sample size: Z < c or Z > c where Z = W µ W nr (nr +1), µ σ W =, W 4 nr (n σ W = r +1)(2n r +1) ; 24 W is approximately normally distributed. 9/44

10 Two related groups Wilcoxon signed ranks test (nonparametric): H0 : population median of differences between paired observations = 0 group 1 group 2 diff abs(diff) rank sign neg pos pos neg pos pos sum of positive ranks = 15; sum of negative ranks = 3 10/44

11 Two related groups Statistical Table 8.1 Critical one- and two-tailed values of T for a Wilcoxon Matched- Pairs Signed-Ranks test, where T = the sum of differences with the least frequent sign and N = the total number of differences with either a positive or negative sign. (T is significant if it is less than or equal to the table value) Critical values for a Wilcoxon signed ranks test N level of significance for a one-tailed test level of significance for a two-tailed test W = 3, n r = 6, c = 0 for two-sided test and α = 0.05 W > c thus we do not reject the null hypothesis /44

12 Two related groups Example SPSS output: Response of serum antigen level to AZT in 20 AIDS patients (Makutch and Parks, 1988) H0 : population median post-treatment serum antigen level = population median pre-treatment serum antigen level Test Statistics a Z Asymp. Sig. (2-tailed) Exact Sig. (2-tailed) Exact Sig. (1-tailed) Point Probability b a. b. 12/44

13 Two related groups The sign test (nonparametric): H 0 : population median of differences between paired observations = parameter value For the hours of sleep example: H 0 : population median difference hours sleep = 0 H 1 : population median difference hours sleep 0 No normality required, no symmetry required Robust to outliers Applicable to numerical and ordinal data Less powerful than Wilcoxon signed ranks test when the population is symmetric since it ignores magnitudes completely 13/44

14 Two related groups The sign test (nonparametric): How does it work? Compute the differences between paired observations and omit differences equal to the parameter value (sample size is reduced to n r ) Count the number of positive and negative differences The test statistic W is the number of positive differences or the number of negative differences, whichever is smaller Reject H0 if: p value<α and p value=p(w or less differences) for one-sided test or p value=2p(w or less differences) for two-sided test; W has a binomial distribution with n r trials and p = 1/2. 14/44

15 Two related groups Example SPSS output: Response of serum antigen level to AZT in 20 AIDS patients (Makutch and Parks, 1988). H0 : population median post-treatment serum antigen level = population median pre-treatment serum antigen level Test Statistics a Exact Sig. (2-tailed) Exact Sig. (1-tailed) Point Probability.004 b a. b. 15/44

16 Two unrelated groups Samples from two independent (unrelated) groups of individuals. Example 1: birth weights of children born to n1 = 40 heavy smokers (group 1) and to n 2 = 42 non-smokers (group 2). Example 2: weights of two groups of children, each child being randomly allocated to receive either a dietary supplement (group 1) or placebo (group 2). 16/44

17 Two unrelated groups Unpaired (two-sample) t-test (parametric): H 0 : two populations have the same means For the birth weight example: H 0 : population mean weight group 1 = population mean weight group 2 H 1 : population mean weight group 1 population mean weight group 2 Reject H 0 if: or sample mean group 1 - sample mean group 2 SEM < c with c > 0. sample mean group 1 - sample mean group 2 SEM > c 17/44

18 Two unrelated groups Variable (e.g., weight) is numerical in each group, is normally distributed in each group with (usually unknown) variances. If population variances are equal but unknown: test statistic follows a Student-t distribution: 5% significance level: SEM = s 1/n1 + 1/n 2 and (n c = t n1 +n 2 2,1 α/2 = t n1 +n 2 2,0.975 (s = 1 1)s 1 2+(n 2 1)s 2 2 ) pooled n 1 +n 2 2 standard deviation of the two groups; s i is the standard deviations sample observations group i; n i is the sample size of group i, with i = 1, 2) 18/44

19 Two unrelated groups If population variances known (σ1 2, σ2 2 ) or sample size very large: test statistic follows a normal distribution (z-test): 5% significance level: SEM = σ 2 1 /n 1 + σ 2 2 /n 2 and c = z 1 α/2 = z = 1.96 If population variances unequal and unknown: test statistic follows a Student-t distribution: 5% significance level: SEM = s 2 1 /n 1 + s 2 2 /n 2 and c = t df,1 α/2 with df = (s2 1 /n 1+s 2 2 /n 2) 2 (s 2 1 /n 1 )2 n (s2 2 /n 2 )2 n /44

20 Two unrelated groups Example SPSS output: Galactose binding measurements for patients with Crohn disease and controls. H0 : population mean measurements Crohn disease = population mean measurements controls Independent Samples Test Levene's Test for Equality of t-test for Equality of Means Variances F Sig. t df Sig. (2- Mean Std. Error 95% Confidence Interval of tailed) Difference Difference the Difference Lower Upper Galactose binding Equal variances assumed Equal variances not assumed /44

21 Two unrelated groups Example SPSS output: Galactose binding measurements for patients with Crohn disease and controls. Independent Samples Test Galactose binding Equal variances assumed Equal variances not assumed Levene's Test t-test for Equality of Means for Equality of Variances F Sig. t df Sig. (2- tailed) Mean Difference Std. Error Difference 95% Confidence Interval of the Difference Lower Upper Equal variances are assumed since F = 0.052, p = we interpret results given in the first row, t(27) = 4.261, p < /44

22 Two unrelated groups Wilcoxon rank sum (two-sample) test (nonparametric): H 0 : two populations have the same medians For the birth weight example: H 0 : population median weight group 1 = population median weight group 2 H 1 : population median weight group 1 population median weight group 2 No normality required but the population distribution of the two groups assumed to have the same shape Applicable to numerical and ordinal data 22/44

23 Two unrelated groups Wilcoxon rank sum (two-sample) test (nonparametric): How does it work? Rank the data ignoring grouping Sum up ranks for each group separately The test statistic W is the smaller of the sum of ranks for group 1 and sum of ranks for group 2 Reject H0 if: Small sample size: W c with c > 0 Large sample size: Z < c or Z > c where Z = W µ W, µ σ W = n 1(n 1 +n 2 +1), W 2 n1 n σ W = 2 (n 1 +n 2 +1), n 12 1 is the sample size of the group that has smaller sum of ranks; W is approximately normally distributed. 23/44

24 Two unrelated groups Mann-Whitney U (two-sample) test (nonparametric): H 0 : two populations have the same medians For the birth weight example: H 0 : population median weight group 1 = population median weight group 2 H 1 : population median weight group 1 population median weight group 2 No normality required but the population distribution of the two groups assumed to have the same shape Applicable to numerical and ordinal data 24/44

25 Two unrelated groups Mann-Whitney U (two-sample) test (nonparametric): How does it work? Rank the data ignoring grouping Sum up ranks for each group separately (R 1 and R 2 ) The test statistic U is the smaller value of U 1 and U 2 : U 1 = n 1 n 2 + n 1(n 1 + 1) 2 R 1 U 2 = n 1 n 2 + n 2(n 2 + 1) R 2 2 where n 1, n 2 are the sample sizes of the groups. Reject H0 if: Small sample size: U c with c > 0 Large sample size: Z < c or Z > c where Z = U µ U, µ σ U = n 1n 2 U 2, n1 n σ U = 2 (n 1 +n 2 +1) ; 12 U is approximately normally distributed. 25/44

26 Two unrelated groups Wilcoxon rank sum test and Mann-Whitney U test give identical results Example SPSS output: Data on diastolic blood pressure (mm Hg) measured in 4 treated subjects and 11 controls. H0 : population median diastolic blood pressure treated subjects = population median diastolic blood pressure controls Test Statistics a Mann-Whitney U Wilcoxon W Z Asymp. Sig. (2-tailed) Exact Sig. (2-tailed) Exact Sig. (1-tailed) Point Probability b a. b. 26/44

27 More than two related groups Related groups: One group of individuals and variable measured on each individual in more than two circumstances. E.g. measurement while taking low dose of a drug, high dose of a drug or placebo. More than two samples of different individuals but linked to each other. Making pairwise comparisons between groups is not efficient because the type I error rate becomes high Better to carry out a single global test to determine whether the means/medians differ in ANY groups. 27/44

28 More than two related groups Friedman s ANOVA (nonparametric): H 0 : all populations have the same medians H 1 : at least one population has a different median Example: we measure the outcome variable x of n individuals at k different conditions or k different time points. No normality required Applicable to numerical and ordinal data 28/44

29 More than two related groups Friedman s ANOVA (nonparametric) How does it work? Rank the data separately for each individual / group of related observations Sum up ranks for each group separately (R j for group j) The test statistic F is F = 12 R 2 j 3n(k + 1) nk(k + 1) Reject H0 if: Small sample size: F c with c > 0 Large sample size: F < c or F > c with c > 0 knowing that F has a chi-square distribution with df=k 1 j 29/44

30 More than two unrelated groups Examples: Samples from 3 independent groups of patients, each with a type of sickle cell disease. For each patient, the steady-state haemoglobin levels are measured. RNA samples from 12 mice of 3 different strains (4 mice/strain). Identify genes that differ in expression levels among these strains. Making pairwise comparisons between groups (e.g. with t-test) is not efficient because the type I error rate becomes high Better to carry out a single global test to determine whether the means/medians differ in ANY groups. 30/44

31 More than two unrelated groups One-way analysis of variance (ANOVA) (parametric): H 0 : all populations have the same means H 1 : at least one population has a different mean (Homogeneity or heterogeneity across populations/groups) Example: the populations means haemoglobin level for each type of sickle cell disease are the same, or at least one is different. Variable (e.g., haemoglobin level, gene expression) is numerical in each group is normally distributed in each group and variances are the same across groups moderate departures from normality may be ignored but unequal variances cannot check homogeneity of variances. Groups are defined by the levels of a single factor (e.g. different sickle cell disease; gender). 31/44

32 More than two unrelated groups One-way analysis of variance (ANOVA) (parametric): We measure the outcome variable x (e.g. haemoglobin level) and compare its mean in the k groups defined by the levels of a single factor (e.g. type of sickle cell disease). The outcome is measured n times in total. The variance of all observations ignoring subdivision into groups (total sample variance) is s 2 = j i (x ij x) 2 /(n 1) One-way ANOVA partitions the sum of squares SS = j i (x ij x) 2 into: Between-groups SS (k - 1 d.f.): SS M = j n j ( x j x) 2 Within-groups SS or residual SS (n - k d.f.): SS R = j i (x ij x j ) 2 x ij is the i observation in j group, x j is the mean of group j, x is the grand mean The amount of variation per degree of freedom is the mean square (MS). 32/44

33 More than two unrelated groups One-way analysis of variance (ANOVA) (parametric): DV IV Level 1 Level 2 Level 3 SSR uses the differences between the observed data and the group means 1 33/44

34 More than two unrelated groups One-way analysis of variance (ANOVA) (parametric): DV IV Level 1 Level 2 Level 3 SSM uses the differences between the grand means and the group means 2 34/44

35 More than two unrelated groups One-way analysis of variance (ANOVA) (parametric): Reject H0 at the 5% significance level if: F = Between-groups SS/(k 1) Within-groups SS/(n 1) = Between-groups MS Within-groups MS > F k 1,n k,0.95 The F test statistic is found by dividing the between group variance by the within group variance. When there are only 2 groups, results of the one-way ANOVA exactly equal to results of t-test. 35/44

36 More than two unrelated groups Example SPSS output: Galactose binding measurements for patients with Crohn disease, ulcerative colitis and controls. H0 : population mean measurements Crohn disease = population mean measurements ulcerative colitis = population mean measurements controls ANOVA Galactose binding Between Groups Within Groups Total df Mean Square F Sig /44

37 More than two unrelated groups Kruskal-Wallis test (nonparametric): H 0 : all populations have the same medians H 1 : at least one population has a different median The population distribution of all groups assumed to have the same shape We measure the outcome variable x (e.g. haemoglobin level) and compare its median in the k groups defined by the levels of a single factor (e.g. type of sickle cell disease). The outcome is measured n times in total. No normality required Applicable to numerical or ordinal data 37/44

38 More than two unrelated groups Kruskal-Wallis test (nonparametric): How does it work? Rank the data ignoring grouping Sum up ranks for each group separately (R j ) The test statistic H is H = 12 Rj 2 3(n + 1) n(n + 1) Reject H0 if: Small sample size: H c with c > 0 Larger sample size: H < c or H > c with c > 0 knowing that H has a chi-squared distribution with df=k 1 j n j 38/44

39 Multiple comparisons Multiple comparisons: compare all different pairwise combinations of the groups With k groups we have k(k 1) 2 pairs of groups to compare There are methods that control for the increased familywise error rate, i.e., make all pairwise comparisons while maintaining the experimentwise error rate at the pre-established α level 39/44

40 Multiple comparisons SPSS Post Hoc Multiple Comparisons option 40/44

41 Multiple comparisons Fisher s least significant difference (LSD) test: does not correct for multiple comparisons, is equivalent to performing multiple t-tests on the data Bonferroni method: for each pairwise comparison α/m is used as a significance level and overall Type I error rate is α; m is the number of all possible comparisons in SPSS: each p-value for Bonferroni test is a p-value for LSD test multiplied by the number of comparisons powerful method for a small number of comparisons 41/44

42 Multiple comparisons Tukey s HSD(Honestly Significance Difference) test: for each pairwise comparison the test statistic Q = x i x k is used where MSw /n i k, x i and x k are the group means we compare, n is the sample size of each group and MS w is the within-groups variance value from, e.g, the ANOVA method we obtained at the first phase correction for unequal sized groups: MSw is divided by m n h = 1/n 1 +1/n /n m powerful method for a large number of comparisons Dunnett s test: makes pairwise comparisons of each group to a control or reference group so we have k 1 comparisons 42/44

43 Multiple comparisons Example SPSS output: Galactose binding measurements for patients with Crohn disease, ulcerative colitis and controls. Dependent Variable: Galactose binding Multiple Comparisons (I) group (J) group Std. Error Sig. Bonferroni Crohn disease Ulcerative colitis Control * Ulcerative colitis Crohn disease Control Control Crohn disease * Ulcerative colitis Dunnett t (2-sided) b Crohn disease Control * Ulcerative colitis Control Multiple Comparisons Dependent Variable: Galactose binding 95% Confidence Interval 43/44

44 Parametric or nonparametric? Parametric approaches: when a variable is normally distributed dependent groups: check normality for variable of differences independent groups: check normality for each variable separately when n > 30 Nonparametric approaches (variable s observations are replaced with their ranks): when a variable is not normally distributed when a variable is ordinal or have many outliers when n is small when median is a better representation of the study 44/44