Chapter 21 Statistical Tests for Ordinal Data. Table 21.1

Transcription

1 Chapter 21 Statistical Tests for Ordinal Data The Mann-Whitney (Rank-Sum) Test In this example, we deal with a dependent variable that is measured on a ratio scale (time in seconds), but its distribution appears to be inconsistent with the use of parametric statistics. In our hypothetical experiment, subjects are asked to find a "hidden" figure (e.g., a drawing of a dog) embedded in a very complex visual stimulus. The amount of time is recorded until they can trace the hidden figure. One group of seven subjects sees a quick flash of the hidden figure (the flash is so fast, it's subliminal) before beginning the task, while the other group of seven sees an equally quick flash of an irrelevant drawing. The amount of time spent by the "primed" group (correct hidden drawing was flashed) can be compared to the "unprimed" group (irrelevant flash) using procedures for ordinal statistics. The amount of time (in seconds) spent finding the figure is shown for each of the 14 subjects in Table Table 21.1 Primed Unprimed First, we rank the data for both groups combined, giving the average ranks for tied measurements, as shown in Table Each rank can be marked with an "A" or a "B" according to which group is associated with that rank. Because the two samples are the same size, it is arbitrary which group is labeled the smaller group (n S = n L = 7); for this example, I will label the primed group (A) as the "smaller" group. Table 21.2 Time Rank Group 8 1 A 10 2 B 12 3 A 20 4 B 22 5 A

2 38 6 B 42 7 A 45 8 B A A B B A B Now we sum the ranks for the primed (A) group. ΣR A = S S = = 48. As a check, we find ΣR B = 57, and note that ΣR A + ΣR B = = 105, which is the same as the sum of ranks 1 to 14 as found by Formula 21.1: N(N +1) 14(15) S R = = = = 105 Looking in Table A.15, under n A = 7, we see that the entry for n B = 7, and significance level =.025, is Because 48 > 36 (or because 57 < 69), the result is not significant at the.05, two-tailed level. The entry for a one-tailed.05 test is 39-66, so the result would not be significant with a one-tailed test, either. The Normal Approximation to the Mann-Whitney Test The amount of ΣRA, which I label as S S (for smaller sum) below, can be converted to a z-score by means of Formula The z-score for the present example is: S z = S -.5( ns )(N +1) (7)(15) = = = = -.57 ns nl(n +1) (7)(7)(15) Although our sample sizes are too small to justify using the normal approximation, it is not surprising that the z-score is very far from statistical significance, given that the sums of ranks for the two groups are not very different. Effect Size for the Mann-Whitney Test To compute the rank biserial correlation coefficient, rg, just subtract the average rank for one group (48/7) from the average for the other (57/7), double that difference, and divide by the total N. For the equal-n case, you can just divide the difference of the two sums of ranks by the square of n: r G = (57 48)

3 / 7 2 = 9/49 =.184, a rather small effect. The Wilcoxon (Signed-Ranks Test) In order to create a matched-pairs design, let us imagine that the subjects in Table 21.1 had actually been paired together based on previous scores on a "field dependency" test, so that each row of Table 21.1 contains a pair of matched subjects who had been randomly assigned to either the primed or unprimed group. The differences between the two conditions have been calculated and added to Table 21.1 to create Table 21.3: Table 21.3 Pair # Primed Unprimed Difference The next step is to rank order the magnitude of the difference scores without regard to their direction (i.e., sign), but next to each rank you should indicate the sign of the difference score with which it is associated, as in Table Table 21.4 Difference Score Rank -2 (-)1-4 (-)2 +5 (+)3 +12 (+)4 +23 (+)5 +29 (+)6 +84 (+)7

4 The final step is to sum the negatively and positively associated ranks, separately. From Table 21.4, ΣR minus = = 3, and ΣR plus = = 25. The value of T is the smaller of the two sums, so T = 3. According to Table A.16, for n = 7, T must be less than or equal to 2 to be significant at the.05 two-tailed level, so the null hypothesis cannot be rejected at that level. However, for a one-tailed test, the critical value of T is 3; because our calculated T equals that critical T, we could declare our results significant at the.05 level with a one-tailed test. The Normal Approximation to the Wilcoxon Test Wilcoxon's T can be converted to a z-score by using Formula 21.5, as shown below for the present example. z = T -.25N(N +1) N(N +1)(2N +1) (7)(8) 3-14 = = 7(8)(15) = = Although the normal approximation would not be used for such a small sample size, the approximation in this case is actually a reasonably good one (as compared to a more exact test). The z-score calculated above is significant at the.05 level for a one-tailed, but not a two-tailed test. If we assume that the two population distributions are similar in form, and we can justify a one-tailed test, we can assert that the median solution time for the primed population is less than for the unprimed population. Note that the Mann-Whitney test failed to even approach a significant difference between the groups, but the extra power gained by matching the subjects led to the (one-tailed) significance of the Wilcoxon test. The Wilcoxon matched-pairs test has an advantage over the Mann-Whitney test only if the scores are fairly well matched. The degree of matching can be measured by means of the Spearman correlation coefficient, r s. Spearman Correlation for Ranked Data Correlation coefficients are often used to describe the reliability of a test, or the matching between sets of scores in a study, without testing any hypothesis. We will use correlation in this way to test the matching in Table Pearson r can be calculated directly for the data in Table 21.1, but we will assume that the distributions of the variables are not compatible with the assumptions underlying the use of Pearson's r. To circumvent these assumptions, we assign ranks to the data separately for each variable, and then apply Pearson's formula to these ranks. That is, we rank the "primed" scores 1 to 7 (giving average ranks to ties), and then rank the unprimed scores 1 to 7. The original data is then replaced by these ranks. In Table 21.5, I have included the original data from Table 21.1 for the sake of clarity, along with the corresponding ranks. Table 21.5 Primed Rank Unprimed Rank D D

5 ΣD 2 = 4.5 Any of the formulas for Pearson's r can be applied directly to the ranks in Table 21.5, and the result would be r s. However, if calculating by hand, it is easier to compute the difference score for each pair of ranks, square each difference, find the sum of the squared differences (ΣD 2 ), and insert this sum into a shortcut formula for Spearman correlation. That is why these differences (D) and squared differences (D 2 ) were included in Table To find r s, take ΣD 2 from Table 21.5, and plug it into the following short-cut formula, in which N equals the number of pairs of scores. 2 6Σ 6(4.5) = 1- D = 1- = 1- N( N -1) 7(48) r s = =.920 As you can see, r s is very high, which indicates that the pairs of scores were very well matched. This high correlation explains the large discrepancy between the results of the Mann-Whitney test (which ignores the matching), and the Wilcoxon test (which uses the matching). Assumptions of Tests on Ordinal Data All three of the tests described in this review are "distribution-free" in that none makes any assumption about the shape of the distribution of the dependent variable. Only the following two assumptions are required. 1. Independent random sampling. This is the same assumption that is made for parametric tests. 2. The distribution of the dependent variable is continuous. This implies that tied ranks will be rare. If there are more than a few ties, correction factors may be needed. When to Use Ordinal Tests There are two major situations that call for the use of ordinal tests: 1. The dependent variable has been measured on an ordinal scale. In some cases, it is not feasible to measure the DV precisely (e.g., the DV is charisma, or creativity), but it is possible to place participants in order of magnitude on the DV, and assign ranks. 2. The dependent variable has been measured on an interval or ratio scale, but the distribution of the DV

6 does not fit the assumptions for a parametric test. The smaller the samples, the less accurate parametric tests become in this situation. The interval/ratio measurements are assigned ranks before applying ordinal tests. Definitions of Key Terms Mann-Whitney Test. Compares two independent groups when the dependent variable has been measured on an ordinal scale. Also called the Mann-Whitney U test (if the U statistic is computed), or sometimes the Wilcoxon-Mann-Whitney test. Wilcoxon matched-pairs test. Replaces the matched t-test when differences scores are not considered precise, but can be rank-ordered. Also called the Wilcoxon T test, because the test statistic is referred to as T. Spearman correlation, r s. This is the result of applying the Pearson correlation formula to two variables when both are in the form of ranks. Kruskal-Wallis test. This test is a direct extension of the Mann-Whitney test that can accommodate any number of independent groups. Because it replaces the one-way ANOVA when the data are in the form of ranks, it is sometimes called the Kruskal-Wallis one-way analysis of variance by ranks. It is also called the Kruskal-Wallis H test, because the test statistic is referred to as H. Friedman test. This test replaces the one-way repeated-measures or randomized blocks ANOVA when the data are in the form of ranks. Practice Exercises 1. In one study, boys were classified as having experienced parental divorce (DB) or not (NDB). The number of fights initiated by each boy during school recess is recorded for a period of three months. The data appear separately for each group of boys below: DB: 3, 5, 0, 9, 1, 7, 4 NDB: 0, 2, 1, 0, 2, 0, 3, 1, 0 a) Perform the Mann-Whitney test for these data, using the normal approximation. Calculate a measure of effect size. b) Perform the Kruskal-Wallis test on these data. Calculate a measure of effect size. Explain the relation between the H you just calculated and the z you found in part a. 2. Perform the Wilcoxon signed-ranks test on the data from Exercise #3 in Chapter 19: a) using the appropriate critical value from Table A.16. Calculate a measure of effect size. b) using the normal approximation formula (despite the tiny N).

7 3. Calculate the Spearman rank correlation coefficient for the data from Exercise #3 in Chapter 19. Does the r S you calculated represent a large effect size? How can you tell?