Chi-Square Test for Goodness of Fit

Transcription

1 Ch-Square Test or Goodness o Ft Scentsts oten use the Ch-square χ test to determne the goodness o t between theoretcal and epermental data. In ths test, we compare observed values wth theoretcal or epected values. Observed values are those that the researcher obtans emprcall through drect observaton. The theoretcal or epected values are developed on the bass o an establshed theor or a workng hpothess. For eample, we mght epect that we lp a con 00 tmes, that we would tall 00 heads and 00 tals. In checkng our hpothess, we mght nd onl 9 heads and 08 tals. Should we reject ths con as beng ar? Should we just attrbute the derence between epected and observed requences to random luctuaton? Consder a second eample: let s suppose that we have an unbased, s-sded de. We roll ths de 300 tmes and tall the number o tmes each sde appears: Face Frequenc Ideall, we mght epect ever sde to appear 50 tmes. What should we conclude rom these results? Is the de based? ull Hpothess The use o the ch-squared dstrbuton s hpothess testng ollows ths process: a null hpothess H 0 s stated, a test statstc s calculated, the observed value o the test statstc s compared to a crtcal value, and 3 a decson s made whether or not to reject the null hpothess. An attractve eature o the ch-squared goodness-o-t test s that t can be appled to an unvarate dstrbuton or whch ou can calculate the cumulatve dstrbuton uncton. The null hpothess s a statement that s assumed true. It s rejected onl when the data has a degree o statstcal condence that the null hpothess s alse, when the level o condence eceeds a pre-determned level, usuall 95 %, that causes a rejecton o the null hpothess. I epermental observatons ndcate that the null hpothess should be rejected, t means ether that the hpothess s ndeed alse or the measured data gave an mprobable result ndcatng that the hpothess s alse, when t s reall true. Ths s an unortunate propert o statstcs. Calculatng Ch-squared For the ch-square goodness-o-t computaton, the data are dvded nto k bns and the test statstc s dened as k O E χ 7 E where O s the observed requenc or bn and E s the epected requenc or bn. Chsquared s alwas postve and ma range rom 0 to. The ch-squared goodness-o-t test s appled to bnned data.e., data put nto classes and s senstve to the choce o bns. Ths s actuall not a restrcton, snce or non-bnned data, a hstogram or requenc table can be made beore generatng the ch-square test. However, the

2 values o the ch-squared test statstc are dependent on how the data s bnned. Another dsadvantage o the ch-square test s that t requres a sucent sample sze n order or the chsquare appromaton to be vald. There s no optmal choce or the bn wdth snce the optmal bn wdth depends on the dstrbuton. Most reasonable choces should produce smlar, but not dentcal, results. One method that ma work s to choose bns that have a wdth o s/3 and lower and upper bns at the sample mean ±6s, where s s the sample standard devaton. For the chsquare appromaton to be vald, the epected requenc should be at least 5. Ths test s not vald or small samples, and some o the counts are less than ve, ou ma need to combne some bns n the tals. Let s appl ths now to the above eamples: Table I: The Con Toss Eample Face O E O E / E Heads Tals Totals χ.8 Table II: The 6-sded De Eample Face O E O E / E Totals χ 0.8 Degrees o Freedom We have seen how to calculate a value or ch-squared, but so ar, t doesn t have much meanng. The ch-square dstrbuton s tabulated and avalable n most tets on statstcs and reprnted here. To use the table, one must know how man degrees o reedom d are assocated wth the number o categores n the sample data. Ths s because there s a aml o ch-square dstrbutons, each a uncton o the number o degrees o reedom. The number o degrees o reedom s tpcall equal to k. For eample, n the de eample, the epected requences or each o the two categores heads, tals are not ndependent. To obtan the epected requenc o tals 00, we need onl subtract the epected requenc o heads 00 rom the total requenc 00. Smlarl, or the de eample, there are s possble categores o outcomes: the occurrence o each o the aces. Under the assumpton that the de s ar, we epect a requenc o 50 or each o the aces, but these agan are not ndependent. Once the requenc count s known or ve o the bns, the requenc o the sth bn s determned, snce the total count s 300. Thus, onl the requences n ve o the s bns are ree to var leadng to ve degrees o reedom or ths eample.

3 Levels o Condence A ch-square table, lke Table III, lsts the ch-squared dstrbuton n terms o d and n terms o the level o condence, α p. Ths ch-squared goodness-o-t method s not wthout rsk; and the data ma lead to the rejecton when n act t s true. Ths s wh we speak o condence. In the con lp eample, the null hpothess s that the requenc o heads s equal to the requenc o tals. In the more general case, we do not requre equal probablt or each o the categores. There are man cases where an epected categor wll contan the majort o tall marks over all other categores one such eample would be a surve enqurng about the publc s choce n an upcomng presdental electon that ncludes all canddates on the ballot. In Table III, the crtcal values o χ are gven or up to 0 degrees o reedom. Four derent percentle ponts n each dstrbuton are gven or p 0.0, 0.05, and The standard practce n the world o statstcs s to use a 95 % level o condence n the hpothess decson makng. Thus, the value o ch-squared that s calculated ndcates a value o p that s less than or equal to 0.05, then the null hpothess should be rejected. In the con-lp eample, ou can toss a con and get 4 heads out o twent lps and nd p Ths would ndcate that such an observaton can happen b chance and the con can be consdered a ar con. Such a ndng would be descrbed b statstcans as not statstcall sgncant at the 5 % level. I one ound 5 heads out o 0 tosses, then p would be somewhat less than 0.05 and the con would be consdered based. Ths would be descrbed as statstcall sgncant at the 5 % level. The sgncance level o the test s not determned b the p value. It s pre-determned b the epermenter. You can choose a 90 % level, a 95 % level, a 99 % level, etc. For the con lp eample, wth one degree o reedom. The χ or the eperment gven n Table I s onl.8. Ths corresponds to a p 0.6, whch s somewhat greater than Thereore, the null hpothess that the de s ar cannot be rejected. The smaller the p-value, the greater s the lkelhood that the null hpothess should be rejected. In the case o the data n Table II or the de, the ch-square value s 0.8, whch corresponds to a 93 % condence level. The de would be consdered ar. Wh p 0.05 or a 95 % Level o Condence used? Long ago, beore the wde-spread avalablt o computers, calculatng p values was somewhat dcult, so the values were tabulated or people to nterpolate the p values. The tables that were most commonl used were publshed b Ronald A. Fsher begnnng n the 930s. These tables were subsequentl reproduced n statstcs books everwhere. In Fsher s books, he argued the level o p 0.05 as the measure o whether somethng sgncant s gong on b statng, The value or p 0.05 or n 0 s.96 or nearl ; t s convenent to take ths pont as a lmt n judgng whether a devaton ought to be consdered sgncant or not. Devatons eceedng twce the standard devaton are thus ormall regarded as sgncant. Usng ths crteron, we should be led to ollow up a alse ndcaton onl once n trals, even the statstcs were the onl gude avalable. Fsher contnued hs dscusson n another part o hs book,

4 I one n twent does not seem hgh enough odds, we ma, we preer t, draw the lne at one n t the % pont, or one n a hundred the % pont. Personall, the wrter preers to set a low standard o sgncance at the 5 percent pont, and gnore entrel all resultss whch al to reach ths level. A scentc act should be regardedd as epermentall establshed onl a properl desgned eperment rarel als to gve ths level o sgncance. Table III. The Ch-Square Dstrbuton α χ d α 0.0 α 0.05 α 0.05 d α 0.0 α 0.05 α Fractonal Uncertant Revsted When a reported value s determned b takng the average o a set o ndependent readngs, the ractonal uncertant s gven b the rato o the uncertant dvded b the average value. For ths eample,

5 ractonal uncertant uncertant average 0.05 cm 3.9 cm ote that the ractonal uncertant s dmensonless the uncertant n cm was dvded b the average n cm. An epermental phscst mght make the statement that ths measurement s good to about part n 500" or "precse to about 0.%." The ractonal uncertant s also mportant because t s used n propagatng uncertant n calculatons usng the result o a measurement, as dscussed n the net secton. Propagaton o Uncertant Let sa we are gven a unctonal relatonshp between several measured varables,, z,,,z What s the uncertant n the uncertantes n,, and z are known? To calculate the varance n, as a uncton o the varances n and we use the ollowng: 8 I the varables and are uncorrelated 0, the last term n the above equaton s zero. We can derve the equaton 8 as ollows: Assume we have several measurements o the quanttes e.g.,... and e.g.,.... Then, the average o and s and Assume that the measured values are close to the average values, evaluatng at those measured values...,. Let, ow, epand about the average values., hgher order terms But, let s take the derence and neglect the hgher order terms: The varance s

6 μ μ μ μ μ μ μ μ Snce the dervatves are evaluated at the average values, we can pull them out o the summaton. Eample: Power n an electrc crcut s P I R. Let I.0 ± 0. A and R 0.0 ±.0 Ω. Determne the power and ts uncertant usng propagaton o errors, assumng I and R are uncorrelated. 0 5 watts I IR R P I P R I R R I I P The uncertant n the power s the square root o the varance. P I R 0.0 ± W I the true value o the power was 0.0 W, and we measured t man tmes wth an uncertant s ± W and Gaussan statstcs appl, then 68% o the measurements would le n the range o 8 to watts More Eamples: In each o the ollowng eamples, the uncertant and the ractonal uncertant are gven. a b and and c

7 and and 4 ote: the ractonal uncertant n, as shown n b and c above, has the same orm or multplcaton and dvson: The ractonal uncertant n a product or quotent s the square root o the sum o the squares o the ractonal uncertant o each ndvdual term, as long as the terms are not correlated. Eample: Fnd the ractonal uncertant n v, where v at where a 9.8 ± 0. m/s and t. ± 0. s. v v a a t t or 3.% otce that snce the relatve uncertant n t.9 % s sgncantl greater than the relatve uncertant or a.0 %, the relatve uncertant n v s essentall the same as or t about 3%. Tme-savng appromaton: "A chan s onl as strong as ts weakest lnk." I one o the uncertant terms s more than 3 tmes greater than the other terms, the rootsquares ormula can be skpped, and the combned uncertant s smpl the largest uncertant. Ths shortcut can save a lot o tme wthout losng an accurac n the estmate o the overall uncertant. The Upper-Lower Bound Method o Uncertant Propagaton An alternatve and sometmes smpler procedure to the tedous propagaton o uncertant law that s the upper-lower bound method o uncertant propagaton. Ths alternatve method does not eld a standard uncertant estmate wth a 68% condence nterval, but t does gve a reasonable estmate o the uncertant or practcall an stuaton. The basc dea o ths method s to use the uncertant ranges o each varable to calculate the mamum and mnmum values o the uncton. You can also thnk o ths procedure as eamnng the best and worst case scenaros. For eample, ou took an angle measurement: θ 5 ± and ou needed to nd cos θ, then ma cos mn cos ± ote that even though θ was onl measured to sgncant gures, s known to 3 gures.

8 As shown n ths eample, the uncertant estmate rom the upper-lower bound method s generall larger than the standard uncertant estmate ound rom the propagaton o uncertant law. The upper-lower bound method s especall useul when the unctonal relatonshp s not clear or s ncomplete. One practcal applcaton s orecastng the epected range n an epense budget. In ths case, some epenses ma be ed, whle others ma be uncertan, and the range o these uncertan terms could be used to predct the upper and lower bounds on the total epense. Use o Sgncant Fgures or Smple Propagato on o Uncertant B ollowng a ew smple rules, sgncant gures can be used to nd the approprate precson or a calculated result or the our most bascc math unctons, all wthout the use o complcated ormulas or propagatng uncertantes. For multplcaton and dvson, the number o sgncant gures that are relabl known n a product or quotent s the same as the smallest number o sgncant gures n an o the orgnal numbers. Eample: 6.6 sgncant gures sgncant gures sgncant gures For addton and subtracton, the result should be rounded reported or the least precse number. Eamples: o to the last decmal place I a calculated number s to be used n urther calculatons, t s good practce to keep at least one etra dgt to reduce roundng errorss that ma accumulate. Then the nal answer should be rounded accordng to the above gudelnes. Uncertant and Sgncant Fgures For the same reason thatt t s dshonest to report a result wth more sgncant gures than are relabl known, the uncertant value should also not be reported wth ecessve precson. For eample, we measure the denst o copper, t would be unreasonable to report a result lke: measured denst 8.93 ± g/cm 3 WROG! The uncertant n the measurement cannot be known to that precson. In most epermental work, the condence n the uncertant estmate s not much better than about ± 50% because o all the varous sources o error, none o whchh can be known eactl. Thereore, to be consstent wth ths large uncertant n the uncertant!

9 the uncertant value should be stated to onl one sgncant gure or perhaps sg. gs. the rst dgt s a. Epermental uncertantes should be rounded to one or at most two sgncant gures. So, the the above result should be reported as To help gve a sense o the amount o condence that can be placed n the standard devaton, Table IV ndcates the relatve uncertant assocated wth the standard devaton or varous sample szes. ote that n order or an uncertant value to be reported to 3 sgncant gures, more than readngs would be requred to just ths degree o precson! When an eplct uncertant estmate s made, the uncertant term ndcates how man sgncant gures should be reported n the measured value not the other wa around!. For eample, the uncertant n the denst measurement above s about 0.5 g/cm 3, so ths tells us that the dgt n the tenths place s uncertan, and should be the last one reported. The other dgts n the hundredths place and beond are nsgncant, and should not be reported: measured denst 8.9 ± 0.5 g/cm 3 RIGHT! An epermental value should be rounded to an approprate number o sgncant gures consstent wth ts uncertant. Ths generall means that the last sgncant gure n an reported measurement should be n the same decmal place as the uncertant. In most nstances, ths practce o roundng an epermental result to be consstent wth the uncertant estmate gves the same number o sgncant gures as the rules dscussed earler or smple propagaton o uncertantes or addng, subtractng, multplng, and dvdng. Cauton: When conductng an eperment, t s mportant to keep n mnd that precson s epensve both n terms o tme and materal resources. Do not waste our tme trng to obtan a precse result when onl a rough estmate s requred. The cost ncreases eponentall wth the amount o precson requred, so the potental benet o ths precson must be weghed aganst the etra cost. Table IV. Relatve Uncertant Assocated wth the Standard Devaton or Varous Sample Szes Relatve Uncert.* Sg. Fgs. Vald Impled Uncertant 7% ± 0% to 00% 3 50% ± 0% to 00% 4 4% ± 0% to 00% 5 35% ± 0% to 00% 0 4% ± 0% to 00% 0 6% ± 0% to 00%

10 30 3% ± 0% to 00% 50 0% ± % to 0% 00 7% ± % to 0% % 3 ± 0.% to % *The relatve uncertant s gven b the appromate ormula: Combnng and Reportng Uncertantes In 993, the Internatonal Standards Organzaton ISO publshed the rst ocal worldwde Gude to the Epresson o Uncertant n Measurement. Beore ths tme, uncertant estmates were evaluated and reported accordng to derent conventons dependng on the contet o the measurement or the scentc dscplne. Here are a ew ke ponts rom ths 00- page gude, whch can be ound n moded orm on the IST webste see Reerences. When reportng a measurement, the measured value should be reported along wth an estmate o the total combned standard uncertant o the value. The total uncertant s ound b combnng the uncertant components based on the two tpes o uncertant analss: Tpe A evaluaton o standard uncertant method o evaluaton o uncertant b the statstcal analss o a seres o observatons. Ths method prmarl ncludes random errors. Tpe B evaluaton o standard uncertant method o evaluaton o uncertant b means other than the statstcal analss o seres o observatons. Ths method ncludes sstematc errors and an other uncertant actors that the epermenter beleves are mportant. The ndvdual uncertant components should be combned usng the law o propagaton o uncertantes, commonl called the "root-sum-o-squares" or "RSS" method. When ths s done, the combned standard uncertant should be equvalent to the standard devaton o the result, makng ths uncertant value correspond wth a 68% condence nterval. I a wder condence nterval s desred, the uncertant can be multpled b a coverage actor usuall k or 3 to provde an uncertant range that s beleved to nclude the true value wth a condence o 95% or 99.7% respectvel. I a coverage actor s used, there should be a clear eplanaton o ts meanng so there s no conuson or readers nterpretng the sgncance o the uncertant value. You should be aware that the ± uncertant notaton ma be used to ndcate derent condence ntervals, dependng on the scentc dscplne or contet. For eample, a publc opnon poll ma report that the results have a margn o error o ± 3%, whch means that readers can be 95% condent not 68% condent that the reported results are accurate wthn 3 percentage ponts. In phscs, the same average result would be reported wth an uncertant o ±

11 .5% to ndcate the 68% condence nterval. Concluson: "When do measurements agree wth each other?" We now have the resources to answer the undamental scentc queston that was asked at the begnnng o ths error analss dscusson: "Does m result agree wth a theoretcal predcton or results rom other eperments?" Generall speakng, a measured result agrees wth a theoretcal predcton the predcton les wthn the range o epermental uncertant. Smlarl, two measured values have standard uncertant ranges that overlap, then the measurements are sad to be consstent the agree. I the uncertant ranges do not overlap, then the measurements are sad to be dscrepant the do not agree. However, ou should recognze that ths overlap crtera can gve two opposte answers dependng on the evaluaton and condence level o the uncertant. It would be unethcal to arbtrarl nlate the uncertant range just to make the measurement agree wth an epected value. A better procedure would be to dscuss the sze o the derence between the measured and epected values wthn the contet o the uncertant, and tr to dscover the source o the dscrepanc the derence s trul sgncant. Reerences Talor, John. An Introducton to Error Analss, nd. ed. Unverst Scence Books: Sausalto, 997. Bard, DC Epermentaton: An Introducton to Measurement Theor and Eperment Desgn, 3rd. ed. Prentce Hall: Englewood Cls, 995. Bevngton, Phllp and Robnson, D. Data Reducton and Error Analss or the Phscal Scences, nd. ed. McGraw-Hll: ew York, 99. Fsher, RA. Statstcal Methods or Research Workers, Olver & Bod Publshers, 958. ISO. Gude to the Epresson o Uncertant n Measurement. Internatonal Organzaton or Standardzaton ISO and the Internatonal Commttee on Weghts and Measures CIPM: Swtzerland, 993. IST. Gudelnes or Evaluatng and Epressng the Uncertant o IST Measurement Results, 994. Avalable onlne: Portons o ths document on measurements and error was moded rom a document orgnall prepared b The Unverst o orth Carolna at Chapel Hll, Department o Phscs and Astronom