Chi-Square Analyses. Goodness of Fit Often, the first introduction to Chi-square is the Goodness-of-Fit model. Consider the following problem:

Transcription

1 Ch-Square Analyses NCCTM Oct. 10, 003 Ch-Square Analyses Ch-square analyss s one of the more complcated analyses n the AP Statstcs currculum. The complcaton s a result of several features: 1) the pluralty of dfferent analyses (Goodness-of-Ft, Independence, Homogenety), ) the more extensve computatons requred, 3) a computatonal formula that makes lttle sense to students, and 4) the focus on dstrbutons rather than means. My experence suggests that the Ch-square computaton s one that students have seen before (usually as Goodness-of-Ft n Bology or Genetcs class). In ths sesson, we wll try to brng some clarty and understandng to the varous Ch-square analyses. Goodness of Ft Often, the frst ntroducton to Ch-square s the Goodness-of-Ft model. Consder the followng problem: The chocolate candy Plan M&M s have sx dfferent colors n each bag: brown, red, yellow, blue, orange, and green. Accordng to the M&M s webste the ntended dstrbuton of the colors s 30% brown, 0% red, 0% yellow, 10% blue, 10% orange, and 10% green. If several bags are opened, and we consder these to be a random sample of M&M s made, how well does ths dstrbuton descrbe the observed M&M s. Suppose we have 70 M&M s from 5 bags, and we have: Color Brown Red Yellow Blue Orange Green Observed Count Expected Count Are these counts consstent wth the stated proportons? Are they what we expect to see, or are they surprsng? We can perform a hypothess test. The null hypothess s that the proportons are as stated. The alternatve hypothess s that at least one proporton s dfferent from those stated. ( ) O E We compute χ =, whch n ths case s E ( ) ( ) ( ) ( ) ( ) ( ) χ = = Does ths value of χ rase questons about the valdty of the null hypothess? ( ) O E If the null hypothess s true, what values of χ = would lkely be E computed and where does ths value of ft n that spectrum of lkely values? Is a typcal value or an atypcal value? We can create a smulaton that wll help answer ths queston, and see how our smulaton compares to the theoretcal result. Danel J. Teague 1 NC School of Scence and Mathematcs

2 Ch-Square Analyses NCCTM Oct. 10, 003 Smulaton The followng program wll smulate ths stuaton and allow us to see how unusual or how typcal s our value of TI-83 Program for Ch-Square Smulaton PROGRAM:MMS PROGRAM:GENCOLOR ClrHome 0 B Dsp "NUM. MMS?" rand A Input M If A.3 Dsp "NUM. SIMS?" 1 B Input S If A>.3 and A.5 ClrHome B ClrLst LSAMP, LCOLOR, LTALLY, LEXPTD, LCHISQ If A>.5 and A.7 For(I,1,S) 3 B Output(1,1," ") If A>.7 and A.8 Output(1,1,S+1-I) 4 B For(C,1,M) If A>.8 and A.9 prgmgencolor 5 B B LSAMP(C) If A>.9 and A 1 6 B seq(x,x,1,6) LCOLOR For(X,1,6) 0 D For(Y,1,M) If LSAMP(Y)=X D+1 D D LTALLY(X) {.3,.,.,.1,.1,.1}*M LEXPTD sum((ltally-lexptd)²/lexptd) LCHISQ(I) If we run ths smulaton 10,000 tmes, we can plot a bar graph of the dstrbuton of Chsquare values under the null hypothess (see Fgure 1). The sample dstrbuton shown below wth the dotted curve llustratng the samplng dstrbuton of Ch-square wth 5 degrees of freedom stretched by a factor of 10,000. From the bar graph n Fgure 1, we see that the value of s qute typcal of the Chsquare values generated when the null hypothess s true. We also see that the dstrbuton of our 10,000 trals matches the contnuous dstrbuton farly well, although t seems that we have a few too many very small values of Ch-square. We can also see from Fgure 1 that we shouldn t consder a value of Ch-square to be unusually large unless t s larger than 10 or 1. If we look at the contnuous dstrbuton, we fnd that 5% of the scores are expected to be larger than when the null hypothess s true. Ths s the crtcal value of Ch-square wth 5 degrees of freedom for a sgnfcance level of Fgure 1: Bar graph of smulaton Ch-square values and Theoretcal Dstrbuton Danel J. Teague NC School of Scence and Mathematcs

3 Ch-Square Analyses NCCTM Oct. 10, 003 Expected Cell Counts > 5 The smulaton also allows us to nvestgate the condtons for usng the Ch-square dstrbuton to approxmate the theoretcal (dscrete) dstrbuton of counts under the null hypothess. Most texts requre all cell counts to have an expected value of at least 5. What happens f ths s not true? Suppose the number n the sample s only 10. In ths case, the largest expected cell count s only 3. The crtcal value of Ch-square for 5 degrees of freedom at the 0.05 level of sgnfcance s We should expect to fnd approxmately 50 of the 1000 runs of the smulaton to have a Ch-square value greater than However, f the number n the sample s 10, we fnd many more smulatons wth Ch-square scores larger than We reject the null hypothess much more frequently than we should. We ran the smulaton of 1000 trals ten tmes each wth N = 10, 0, 30, 40, 50, 60 70, and 100. As N ncreases, more of the expected counts are 5 or greater. Once N = 50, all expected counts are at least 5, so we would expect to fnd close to 5% of the 1000 trals wth a Ch-square value greater than Table 1 gves the number of trals over the crtcal value for each of the 10 repettons. N = 10 N = 0 N = 30 N = 40 N = 50 N = 60 N = 70 N = Mean Table 1: The Number of Ch-square Scores above Notce that as the number of cells whose expected value exceeds 5 ncreases, the mean number of Ch-square values larger than decreases to around 50 per 1000 trals. Once the number of M&M s sampled exceeds 50, all of the means reman essentally constant at around 50, as expected. ( ) O E Understandng the Formula χ = E ( O E ) Just what s beng measured n the computaton of? How can we make E sense out of ths expresson? We are nterested n comparng our observed counts wth the expected counts, so ( O E) certanly makes sense. However, there are three problems wth just summng up the terms ( O E). 1) The frst problem s one of scale or sze. Suppose O E = 4? Is ths a large error or a small error? If E = 5 and O = 9, and error of 4 s qute large. However, f E = 5000 and O = 5004, an error of 4 s qute small. The absolute dfference n O and E s often not the most Danel J. Teague 3 NC School of Scence and Mathematcs

4 Ch-Square Analyses NCCTM Oct. 10, 003 mportant measure. We have a standard way to deal wth ths. We use the relatve error. That s, O E we consder. E ) But addng up all of the relatve errors leads to a second problem, the problem of sgn. Some of the errors are postve and some are negatve, so the total sum of the relatve errors wll be zero. There s also a standard way to handle ths dffculty. We square the terms O E E. pror to addng them. So we are nterested n 3) But ths also produces a dfferent problem of scale. Suppose the squared relatve error s 1/5. Is ths a large error or a small error? If the expected count s 5, then we have a small error (1 out of 5) and so our squared relatve error of 1/5 should not count too much aganst the null hypothess. If the expected count s 5000, we are off by qute a lot (1000 out of 5000), so our squared relatve error of 1/5 should count a lot aganst the null hypothess. So some values of 1/5 should count more than other values of 1/5. We also have a standard way to handle ths dffculty. We use a weghted sum. We weght the squared relatve error by the expected count sze, so we end up wth whch s our standard computatonal formula. ( O E ) O E E = E E, Tests of Independence and Homogenety There are two forms of the Ch-square test that often get confused, the Ch-square test of ndependence and the Ch-square test of homogenety. The test of ndependence attempts to determne whether two characterstcs assocated wth subjects n a sngle populaton are ndependent. The test of homogenety attempts to determne whether several populatons are smlar (homogeneous) wth respect to some varable. We can use Problem # from the 1999 AP exam as an example to compare these two tests. In ths problem, a random sample of 00 hkers was taken. The hkers were asked f they would walk uphll, downhll, or reman where there were f they became lost n the woods. They hkers were classfed as ether a Novce or an Experenced hker. The queston asks f there s an assocaton between the responses Uphll, Downhll, and Reman, and the classfcaton as Novce or Experenced. Independence As stated, ths s a Ch-square test of ndependence snce a sngle sample from the populaton of hkers was taken, and the ndvduals sorted nto the cells classfed as Novce/Experenced and Uphll/Downhll/Reman. The null hypothess s that there s no assocaton between the two varables. As n all hypothess tests, we use the null hypothess to compute the expected values. The table of counts looked lke ths: Uphll Downhll Reman Novce Experenced Table : Counts for Test of Independence Danel J. Teague 4 NC School of Scence and Mathematcs

5 Ch-Square Analyses NCCTM Oct. 10, 003 If the null hypothess s true, how would the hkers be sorted nto the 6 cells? We need to determne the expected counts n each cell. To determne the expected counts, consder the followng three questons. 1. If a hker from ths sample were selected at random, what s the probablty that they would be a Novce? Answer: p(n) = 10/00. Also note that p(e) = 80/00.. If a hker from ths sample were selected at random, what s the probablty that they would go Uphll? Answer: p(u) = 30/00. Also note that p(d) = 80/00 and p(r) = 90/ Based on these two probabltes, f the level of hkng experence and the drecton a hker would travel f lost are ndependent, what s the probablty that a hker selected at random would be a Novce who would head Uphll? Answer: p(n and U) = p(n)*p(u) = = = The assumpton of ndependence allows us to compute the expected number to be found n ths cell of our table. Snce there are 00 hkers altogether, f the row and column varables 9 are ndependent, we would expect to see 00 = 18 hkers n ths cell. We actually have hkers n ths cell. We repeat ths calculaton for each cell n the table and ask the queston: Are these observed counts consstent wth the expected counts computed under the assumpton of ndependence? Notce that ths computaton s equvalent to, the 00 ( row count)( column count) total, whch s gven by most texts as the computatonal formula. Observatons Uphll Downhll Reman Novce Experenced Expectatons Uphll Downhll Reman Novce = = = 54 Experenced = 1 00 = 3 00 = 36 Table3: Computng Expected Counts Under the Assumpton of Independence Homogenety Now, suppose nstead that a random sample of 10 Novce hkers was taken from a populaton of Novce hkers and a random sample of 80 Experence hkers was taken from a populaton of Experenced hkers. Each member n the two samples was asked about the drecton they would travel when lost. Ths s a test of homogenety snce the are two Danel J. Teague 5 NC School of Scence and Mathematcs

6 Ch-Square Analyses NCCTM Oct. 10, 003 populatons (Novce and Experence) beng classfed on one varable (drecton when lost). Suppose the results were the same as those above and the table of counts s shown below. Uphll Downhll Reman Novce Experenced Table : Counts n Example Problem The null hypothess for ths test of homogenety s that the proportons of hkers fallng nto the three drecton categores are the same for Novce and Experenced hkers. We use ths null hypothess to compute the expected values. Snce ths s a dfferent null hypothess from the test of ndependence, we would expect our computatons to dffer, and they do. To fnd the expected value of Novce-Uphll, we note that there were 00 hkers altogether, and 30 of them would travel uphll. So the proporton of hkers who would travel uphll s 30/00. The null hypothess s that ths proporton of Novce hkers and ths proporton of Experence hkers would travel uphll. It s the same for both. There are 10 Novce hkers, so the expected number of hkers n the Novce-Uphll cell s = 18 and the expected 30 number of hkers n the Experenced-Uphll cell s 80 = 1. Contnung n ths manner, we complete the expected counts n the table. Observatons Uphll Downhll Reman Novce Experenced Expectatons Uphll Downhll Reman Novce = = = 54 Experenced = 1 80 = 3 80 = 36 Table 4: Computng Expected Counts Under the Assumpton of Homogenety Notce two thngs: 1) the computatons for determnng the expected number n each cell for tests of ndependence and for homogenety are dfferent, snce they are derved from the dfferent null hypotheses. ) the results of these computatons are the same. Snce the results of the computatons are the same, the dstncton between tests of ndependence and tests of homogenety s often consdered academc n an ntroductory course lke AP Statstcs. Degrees of Freedom There were 00 hkers n the sample, 10 classfed as beng Novces and 80 classfed as Experenced. We also know that 30 of the hkers answered they would move Uphll, 80 that they Danel J. Teague 6 NC School of Scence and Mathematcs

7 Ch-Square Analyses NCCTM Oct. 10, 003 would move Downhll, and 90 that they would Reman where they were. We descrbe ths nformaton as knowng the margnal totals. When computng the expected values, once we know that there are 18 Novces expected to answer go Uphll, we know wthout computng that there must be 1 Experenced hkers expected to answer go Uphll (snce there were 30 gong uphll altogether). Also, once we have determned that there are 48 Novces expected to answer go Downhll, we know there must be 3 Experenced hkers expected to gve that answer (snce there were 80 n total who answered Downhll). Moreover, the expected number of Novces answerng Reman must be 54, snce there were 10 Novces and 18 and 48 have already been allocated. In a smlar manner, we know the expected number of Experenced hkers who answer Reman must be 36, snce there were 80 n total and 44 have been allocated. In short, knowng only the expected values n two cells allows us to complete the table of expected counts. We say there are two degrees of freedom and our Ch-square statstcs wll df = row 1 column 1. have degrees of freedom. Ths s classcally stated as ( )( ) Apocryphal Hstorcal Note: The Ch-square statstc was frst developed by Karl Pearson about Pearson knew what the Ch-square dstrbuton looks lke, but he was unsure about the degrees of freedom. About 15 years later, Fsher got nvolved. He and Pearson were unable to agree on the degrees of freedom for the two-by-two table, and they could not settle the ssue mathematcally. Pearson beleved there was 1 degree of freedom and Fsher 3 degrees of freedom. They had no nce way to do smulatons, whch would be the modern approach, so Fsher looked at lots of data n two-by-two tables where the varables were thought to be ndependent. For each table he calculated the Ch-square statstc. Recall that the expected value for the Ch-square statstc s the degrees of freedom. After collectng many Ch-square values, Fsher averaged all the values and got a result he descrbed as embarrassngly close to 1. Ths confrmed that there s one degree of freedom for a two-bytwo table. Some years later ths result was proved mathematcally. Danel J. Teague 7 NC School of Scence and Mathematcs