Sample Size to Control Type II Error Probability

Transcription

1 NCC501 Sttistics for Mgemet Smple Size to Cotrol Type II Error Probbility Hypothesis tests re usully desiged with oe gol: mke sure tht the Type I Error Probbility is α or lower. This does ot give y specific protectio gist Type II Error, cceptig the ull hypothesis whe it is flse. Clcultig the Type II Error probbility is source of gret cofusio, d for this reso most people tret it s fter-thought i hypothesis testig. As cosequece, if the ull hypothesis is ot rejected, we re left i the ucomfortble situtio of ot kowig whether H 0 is true, or whether it is flse but the smple ws too smll to give relible test. I the ltter cse, cceptig H 0 would be Type II error. The key to cotrollig Type II error is the smple size. This ote presets formuls tht pproximte how lrge the smple must be. To use these formuls you must mke judgmet: You must specify vlue of popultio prmeter (either µ or p tht represets the ltertive hypothesis, oe tht is differet eough from H 0 to mtter. This is prcticl issue d hs othig to do with sttistics. For exmple, o oe cres if bottlefillig mchie is off by verge of ouces per sixtee-ouce bottle. However, if tht gp were 0.1 ouces someoe might be very cocered. It is your job to decide how lrge the gp must be to ctully mke differece i the rel situtio. The gp betwee H 0 d H is the mi determit of smple size. If the gp is rrow, we eed lrge smple to tell us which hypothesis is true. If the gp is wide, much smller smple will suffice. Simply put, it tkes more iformtio to distiguish betwee very similr ltertives th very differet oes. The formuls for clcultig the pproprite smple size deped o kowig how much vribility there is i the popultio. This presets chicke-d-egg dilemm: you c t clculte the ecessry smple size util you hve estimte of vribility, but you c t get estimte util you collect smple. A ofte-used solutio to this dilemm is to do smll pilot study to estimte those vlues. Aother method is to mke educted guess. I either cse, the resultig smple-size clcultio must be treted s pproximtio. The beefit derived from the smple-size clcultio comes i the iterprettio of the results: If the ull hypothesis is ot rejected, you hve strog evidece tht either H 0 is true, or it is flse by mout too smll to mtter from prcticl stdpoit. The smple-size formuls re bsed o the orml probbility distributio, d certi other ssumptios tht my be pproximtios. Use them to determie the order of mgitude of the smple size, but do ot tret them s exct. For exmple, if the recommeded smple size is 34.5, do ot worry bout whether to use 34 or 35. Roud up to iteger, d cosider icresig the smple bit more to be coservtive.

2 Smple Size to Cotrol Type II Error Probbility p. Oe-Smple Test for Me (File: SmpSize.xls, Sheet: 1-Me µ is the popultio me. µ 0 is the vlue of µ uder the ull hypothesis. µ is vlue of µ tht mkes H true d is just differet eough from µ 0 to mtter. For -til test the vlue of µ could be o either side of µ 0. Use either oe i the * formul. s is the stdrd devitio estimted from the smple. If the popultio vlue (σ is vilble, use it isted of s. This formul for * ssumes tht σ hs the sme vlue uder H s uder H 0. (1 ( z + z β s 0 α Use the Norml Distributio for z α d z β. ( µ µ For two-til test use α/. Exmple: A bottle-fillig mchie hs bee set to produce verge of 16.3 ouces per bottle, i the log ru. Leglly, every bottle must coti t lest 16 ouces. However, the fillig process hs stdrd devitio of 0.06 ouces. If the mchie were set t 16, hlf of the bottles would be uderfilled. They set it t 16.3 to mke sure tht the rte of uderfilled bottles is less th oe i millio. Sometimes the mchie goes out of djustmet. Whe the log-ru verge drops to 16.7 ouces per bottle, the rte of uderfilled bottles exceeds 3 per millio, situtio tht mgemet cosiders serious eough to stop the fillig process d redjust the mchie, lthough the lost productio is substtil cost. Their curret smplig pl is to test 36 bottles d stop the mchie if the smple s verge is below give vlue. However, they re ot sure tht this pl gives error probbilities tht re low eough. Mgemet wts to be 95% certi tht the mchie will be stopped whe its log-ru verge is 16.7 or lower, d to be 95% certi of NOT stoppig the mchie whe its log-ru verge is ctully Is the smple size of 36 sufficiet? This is oe-til test becuse they stop the mchie oly whe the smple verge is below specified level. H 0 is tht the mchie is ruig properly, so µ 0. = 16.3 H is represeted by µ = 16.7, which is fr eough below 16.3 to mtter. The stdrd devitio is Sice it describes the process rther th prticulr smple, it is popultio vlue, σ. To chieve 95% certity of voidig both kids of errors, α=0.05 for oe-til test, so use z α = 1.645, d β = 0.05, so use z β = ( z + z ( µ β s 0 α ( µ = = 43.3 Coclusio: To chieve 5% probbilities for Type I d Type II errors whe the differece betwee i djustmet d out of djustmet is 0.03, the smples should iclude t lest 44 bottles rther th 36.

3 Smple Size to Cotrol Type II Error Probbility p. 3 Two-Smple Test for Mes (File: SmpSize.xls, Sheet: -Mes µ 1 µ is the differece betwee two popultio mes. d 0 is the vlue of µ 1 µ uder the ull hypothesis (usully zero. d is vlue of µ 1 µ tht mkes H true d is just differet eough from d 0 to mtter. For -til test the vlue of d could be o either side of d 0. Use either oe i the * formul. s 1 d s re the stdrd devitios from ech smple. If the popultio vlues (σ 1 d σ re vilble, use them isted of s 1 d s. ( ( z + z ( s + s α β 1 (d d0 Use the Norml Distributio for z α d z β. For two-til test use α/. The recommeded smple sizes re 1 =* d =*. Exmple: The customer service deprtmet of lrge corportio hs begu progrm to bechmrk their service qulity gist their competitors. Their first effort ws to mesure how log it tkes to rech customer service represettive t their 800 umber. A pilot study ws crried out. First they looked t their ow system. Bsed o smple of 30 clls, the verge time ws.5 miutes d the stdrd devitio ws 0.9 miute. Cllig oe of their competitors 30 times resulted i verge of.7 miutes with stdrd devitio of 1.1 miutes. After creful cosidertio, the bechmrkig tem decided tht they should gurd gist mkig error of more th 0.3 miute. Tht is, if the log-ru differece i times were relly 0.3 miute, they wt to be 99% certi tht their study will mke the correct coclusio, which would be tht the compies relly differ. However, they wt to be eqully creful to drw the correct coclusio if the log-ru verge times do ot differ. This is two-til test becuse they re oly skig if they differ from their competitor. The ull hypothesis is o differece so d 0 = 0. For the ltertive hypothesis, differece of 0.3 or more is importt, so d = 0.3. Sice they wt 99% certity of o error, both error probbilities re to be 1%. Thus, α = 0.01 for two-til test, so z α/ =.576. Also, β = 0.01, z β =.36. From the pilot study we hve prelimiry estimtes to use i the smple size formul: s 1 = 0.9 d s =1.1. ( zα + zβ ( s1 + s ( ( (d d0 = = ( Coclusio: To chieve 1% for both error probbilities they should collect smples of t lest 540 from ech popultio.

4 Smple Size to Cotrol Type II Error Probbility p. 4 Oe-Smple Test for Proportio (File: SmpSize.xls, Sheet: 1-Prop p is the popultio proportio. p 0 is the vlue of p uder the ull hypothesis. p is vlue of p tht mkes H true d is just differet eough from p 0 to mtter. For -til test the vlue of p could be o either side of p 0. Choose the oe tht is closest to 0.5. The formul relies o the orml pproximtio to the biomil, so be sure to verify tht *p 5 d *(1-p 5 for both p 0 d p fter you use it. (3 α 0 0 β Use the Norml Distributio for z α d z β. (p p0 For two-til test use α/. Exmple: The presidet wts to be iformed whe her true pprovl rtig differs by more th percetge poits from 60%. She does ot wt to be otified of shift uless the evidece is quite strog. However, she lso does ot wt to be i the embrrssig situtio of ot hvig bee otified whe rel shift hs occurred. The stff is bout to commissio ew survey of 100 voters. This is two-til test becuse otifictio is requested wheever there is chge i either directio. Uless otified otherwise, she ssumes 60% pprovl, so p 0 = 0.6. Becuse H 0 is two-tiled, the ltertive hypothesis could hve either p = 0.6 or p = 0.58; we use p = 0.58, the oe closer to 0.5. No vlues re give for error probbilities. We will use 5% so tht z α/ = d z β = ( z p (1- p + z p (1- p ( zα p0(1- p0 + zβ p (1- p ( ( (.4 (p p 0 = = Coclusio: To chieve 5% error probbilities whe the differece betwee curret pprovl d ew pprovl is 0.0, smple of t lest 7851 observtios is eeded.

5 Smple Size to Cotrol Type II Error Probbility p. 5 Two-Smple Test for Proportios (File: SmpSize.xls, Sheet: -Prop s p 1 p is the differece betwee two popultio proportios. d 0, the vlue of p 1 p uder the ull hypothesis, is ssumed to be zero. d is vlue of p 1 p tht mkes H true d is just differet eough from zero to mtter. p 1 d p re the proportios clculted from the smples. 1p1 + p p = is the pooled estimte. If you hve o estimte of p, use educted guess. 1 + The formul relies o the orml pproximtio to the biomil, so be sure to verify tht * p 5 d *(1- p 5 fter you use it. (4 z α p(1- p - 0.5d p(1- p + z β d Use the Norml Distributio for z α d z β. For two-til test use α/. The recommeded smple sizes re 1 =* d =*. Exmple: Cosumers Uited is testig to see whether there is differece i the results from two pollsters. Ech pollster smpled 1000 voters, skig would you would vote for the curret presidet if the electio were held tody? Oe pollster s result ws 37% d the other ws 43%. Are the smples lrge eough to miti 5% or lower probbilities for both error types? This is two-til test becuse they re oly skig if the pollig methods differ. The ull hypothesis is o differece we c use the method bove. CU hs specified tht differece of 0.03 or more is importt, so tht vlue is p 1 p for H. For two til test, z α = z 0.05 =1.96 d z β = z 0.05 = First clculte the pooled vlue of the smple proportio, p: p = 1p p 1000( (0.43 = = The clculte the recommeded smple size: z α p(1- p - 0.5d p(1- p + z β d 1.96 (.4( (.4(.6-0.5(.03 = = Coclusio: Sice this is much lrger th the published smples, if Cosumers Uited uses the smples of 1000 to test the differece betwee the two pollsters t α=0.05, they fce Type II error probbility tht is much lrger th 5%.