Lecture 9: the χ 2 -test

Transcription

1 Lecture 9: the χ 2 -test S. Massa, Department of Statistics, University of Oxford 19 January 2016

2 Hypothesis Testing 1. Begin with a research hypothesis (this is usually the alternative hypothesis) 2. Decide significance level, e.g. 5%; 3. Set up the null hypothesis (roughly the opposite of the research hypothesis); 4. Calculate a test statistic from your data; 5. Calculate the p-value: the probability under the null hypothesis of observing something at least as extreme as your observation. 6. Reject the null if the p-value is less than the significance level, 5%.

3 The Z-test Roughly the Z-test is given by Z = observed expected. standard error Example (Population mean) Suppose X 1,, X n are n i.i.d. normal samples with known SD σ. H 0 : mean = µ; H 1 : mean µ; Then with X the sample mean, and since SE = σ/ n Z = X µ σ/ n N(0, 1). (1)

4 The Z-test Example (Test for a proportion) When testing the probability of success in n trials H 0 : P(success)= p 0 ; H 1 : P(success) p 0 ; Under the null hypothesis the expected proportion of successes is p 0, the standard error is p 0 (1 p 0 )/n and the test statistic is Z = proportion of successes p 0 p0 (1 p 0 )/n N(0, 1).

5 Z-test recap Some good properties of the Z-statistic computed from the data (definition of statistic); extreme values correspond intuitively to failure of the null; under the null hypothesis we know the distribution of Z; very good for testing whether the mean of quantitative data could have a certain value. But what about categorical/qualitative data?

6 Suicide by Month of Birth Month Prob Female Male Total Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total Mean Table: Suicides by month of birth in England and Wales

7 Suicide by Month of Birth Of course different months of birth have different numbers of suicides, due to chance variation. Is there an underlying relation? Is it more likely that one commits suicide if they were born in a particular month? It seems there are slightly more suicides among those born in March, April and May. This will be our research hypothesis. H 0 : a suicide is equally to be born on any day of the year, 1/ Does the z-test work in this setting?

8 Does the z-test Work in This Setting? One choice is to group March to June into Spring, and Other for all the other months. We can now test for the proportion of spring born suicides. Then 9421 spring born, and other. 122 days in the spring category H 0 : p 0 = Probability of a suicide being born in spring = 122/ = Expected = p 0 = 8980, SE = p 0 (1 p 0 )n = Then the z-statistic becomes Z = observed expected SE = = 5.71, giving a p-value of less than 10 8.

9 Problem with Using the z-test with More than Two Levels The problem is that we made a choice about how to split up the months. This may give one the opportunity to choose the particular splitting that proves the desired result. This problem will always appear whenever there are more than two levels in our categorical variables (12 months). We need a method that deals with all categories simultaneously.

10 Goodness of Fit Tests Observe n samples of a categorical variable with k levels. H 0 : the categories have the probabilities p 1,..., p k. Under the null hypothesis, since there are n observations the expected counts in the categories are np 1,..., np k. Our statistic should measure how far the observed counts n 1,..., n k deviate from the expected ones np 1,..., np k. Add up (n i np i ) 2. Getting 1 rather than 10 in a category should count more than getting 1010 while expecting So we need to normalise. Therefore we get the χ 2 -statistic X 2 := (n 1 np 1 ) (n k np k ) 2 np 1 np k = (observed expected) 2. expected

11 The χ 2 Distribution Straightaway we know that (i) X 2 is a statistic: it can be computed from the data; (ii) large values imply that the null hypothesis is likely not true. (iii) do we know its distribution? YES, but approximately for large n. This distribution is called the χ 2 distribution and is parameterized by a positive integer, the number of degrees of freedom(d.f.) How do we figure out the degrees of freedom?

12 The Number of Degrees of Freedom The number of degrees of freedom depends on: the number of categories; the form of the null hypothesis, specifically the number of parameters. In general degrees of freedom = #categories #parameters fitted from data 1. n large enough: expected count in each category at least 5. If not, merge categories together.

13 Properties of χ 2 The χ 2 distribution with d degrees of freedom is a continuous distribution. It has mean d and variance 2d. Distributions right-skewed but skew drops as d grows. For large d becomes close to normal with mean d and variance 2d.

14 Some Plots Figure: χ 2 distributions with 0.5, 3, 5, and 10 degrees of freedom.

15 Table d.f. P = 0.05 P = If more than 60 degrees of freedom approximate with N(d, 2d). Find p-value for observed X 2 using Z = X2 d 2d N(0, 1), and the normal table.

16 Assumptions for the χ 2 Goodness of Fit Test Simple Random sample: The sample is taken from a fixed population where each individual has equal chance of being selected. Independence: The observations are independent. Sample size: We have a sufficiently large sample size for the whole table and also for the individuals cells: each cell must have expected count at least 5. The sample size assumption goes back to the central limit theorem: here under the null hypothesis you are sampling from a multinomial distribution. You need adequate expected cell counts for the CLT to kick in.

17 Fixed Distribution: a Fair Die Given a standard die we want to test if it s fair. H 0 : each face has probability 1/6. If it is fair then each side has probability 1/6. Roll it 60 times. Expected count 10 for each side. Compute the χ 2 statistic Observed Expected X 2 (16 10)2 (15 10)2 (5 10)2 = = = 15.4

18 Fixed Distribution: a Fair Die To compute the p-value we need to know the number of degrees of freedom. There are 6 categories and no parameters so d.f. = = 5. Therefore comparing X 2 = 15.4 with and we reject the null hypothesis at both the 5% and the 1% significance level.

19 Parametric Families Sometimes we want to know if the data came from a whole family of distributions, for example from the Binomial or Poisson distribution. H 0 : the data comes from a Poisson(µ) distribution, for some µ. At its current state, it s impossible to check the hypothesis. We don t know µ and thus we cannot calculate probabilities under the null hypothesis. First use the data to estimate the parameter. This gives you a concrete null hypothesis to test.

20 Example (Deaths from Horsekicks) Deaths Frequency Total=200 Table: Deaths from horsekicks in Prussian army (Preece, Ross and Kirby, 1988) We want to test H 0 : The observations come from a Poisson distribution. Recall Poisson has a single parameter µ equal to its mean. Approximate the mean with the sample mean: X = 1 ( ) = = 0.61.

21 Example (Deaths from Horsekicks) Now test: H 0 : The observations come from a Poisson(0.61). Construct the expected table: What is the expected frequency of zeroes in 200 samples from a Poisson distribution? expected frequency of zeroes = 200 P (zero) = 200 e = = !

22 Computing the Expected Row expected frequency of zeroes = 200 P (zero) = 200 e = = expected frequency of 1 = 200 e = = 66.29, 1! and so on until we arrive at Expected freq Observed freq The expected frequency for 2 or more is less than 5 so we combine the last three columns Expected freq Observed freq !

23 We then calculate X 2 = ( )2 ( )2 ( ) = The final table has three categories, so we would have 2 degrees of freedom, but we have also estimated one parameter from the data, so that leaves 1 degree of freedom. Consulting the table the critical value for the χ 2 distribution with 1 degree of freedom is 3.84 at 5% and 6.63 at 1%. Conclusion: Since our χ 2 statistic is 0.07 which is less than both critical values, we retain (do not reject) the null hypothesis that the data comes from a Poisson distribution.

24 The Normal Distribution Suppose we are given the heights of 100 students in centimetres Height (cm) Frequency and you want to test whether it comes from a Normal distribution. H 0 : The data follows N(µ, σ 2 ), for some µ and σ 2. H 1 : The data do not follow a Normal distribution. How do you compute the sample mean and sample standard deviation here? Replace each bin with its midpoint, e.g for , for [161,166], and so on up to for [ ].

25 Estimate the Parameters Like with the Poisson distribution, we use the data to estimate the parameters µ and σ 2. Use the sample mean and sample variance. x = 172 s 2 = H 0 : The data follows N(172, ). H 1 : The data do not follow N(172, ).

26 Compute Expected counts in Each Bin I Our observations are provided in bins, so what we can do is compute the expected count in each bin from N(172, ). So first compute the probability of each interval under the null hypothesis, e.g. let Y N(172, ), and Z N(0, 1) Replace the first interval with (, 160) ( Y ) P (Y 160) = P = P (Z 1.61) =

27 Compute Expected counts in Each Bin II For the second interval P (160.5 Y 166.5) ( = P Y ) = P ( 1.61 Z 0.77) = For the last interval replace [185, 190] with [184.5, ). Multiply each probability with 100, the total number of observations.

28 Test Statistic I Height (cm) prob Expected Observed Check: All categories have expected 5? No, so combine last two columns to get Height (cm) Expected Observed

29 Test Statistic II The χ 2 statistic is then X 2 = (5.4 6) ( ) = The number of degrees of freedom is 5 (categories)-1-2 (for the number of parameters estimated from the data)=2. Critical value for χ 2 (2) is 5.99 at the 5% confidence level. Conclusion: At the 5% confidence level, there is not enough evidence to reject the null hypothesis.

30 Contingency Tables Table: NHANES handedness data for Americans aged Men Women Total right-handed left-handed ambidextrous Total T:h This type of table is known as a contingency table. The data seem to suggest that women tend to be right-handed more than men. Let s test this. One can use the χ 2 statistic to test whether two variables are independent.

31 χ 2 Test of Association We have two variables, one for the gender, one for being right or left-handed. Our research hypothesis implies that gender has an effect on whether one is right- or left-handed. H 0 : The two variables are independent. H 1 : The two variables are associated. We will use the χ 2 test: we have the observed counts, but what are the expected counts under the null hypothesis? The null hypothesis implies that the two variables are independent. This means that and so on. P (man left-handed) = P (man) P (left-handed),

32 χ 2 Test of Association But we still don t know P (man) or P(left-handed): estimate from the data. No. of men in sample P (man) = = 1067 No. of participants 2237 = No. of lhanded in sample P (left-handed) = = 205 No. of participants 2237 = This gives us the expected table: e.g. P (man) P (left) 2237 = Men Women right-handed left-handed ambidextrous Table: Expected table

33 Computing the χ 2 Statistic Then X 2 = ( ) ( ) But how many degrees of freedom? + + (8 15)2 15 = 12.4.

34 d.f. for Test of Association I We have two categorical variables one with 2 levels, one with 3 levels. For gender, we estimated the probability of the event {man}, while the other one follows immediately since there are only two possibilities. For handedness, we had to estimate the probability of two of the three levels, the last one following since the probabilities must sum to 1. Thus for each variable, the number of parameters we estimated is one less the number of levels.

35 d.f. for Test of Association II The number of categories is 3 2 = 6. Therefore d.f. = = 2. Remark In general if we have two variables with r and c levels respectively d.f. = r c 1 (r 1) (c 1) = (r 1) (c 1). Conclusion: Since the critical value at the 1% confidence level is 9.21 and our statistic is 12.4 there is enough evidence to reject the null hypothesis. There is association between gender and being right or left-handed.

36 Recap I χ 2 distribution: parametrized by degrees of freedom; Goodness of fit test: Given n sample of a categorical variable with k levels we test the null hypothesis H 0 : the categories have probabilities p 0,..., p k. If observed counts are n 1,..., n k then X 2 := (n 1 np 1 ) 2 np (n k np k ) 2 np k = (observed expected) 2 expected d.f. = #categories #parameters fit from data 1. Assumptions: simple random sample, independence, expected cell count 5;

37 Recap II χ 2 test of association: two categorical variables summarized in a contingency table H 0 : the two variables are independent. H 1 : the two variables are associated. If variables have r and c levels resp. d.f. = (r 1) (c 1).