Standard Errors and Confidence Intervals

Transcription

1 Stadard Errors ad Cofidece Itervals Itroductio I the documet Data Descriptio, Populatios ad the Normal Distributio a sample had bee obtaied from the populatio of heights of 5-year-old boys. If we assume that this variable has a Normal distributio (a assumptio that is, i fact, etirely reasoable) the it will have a populatio mea, μ, whose value is very likely to be of iterest. As this is a populatio parameter we will ever kow its true value, because we will ever have a complete eumeratio of the populatio. Cosequetly we will have to cotet ourselves with kowig what we ca about μ o the basis of radom samples draw from the populatio. The atural way to estimate μ is to compute the mea, m, of the sample ad say that this value is our estimate of μ. The mea of the sample of 99 heights i the sample give i Data Descriptio, Populatios ad the Normal Distributio is cm. Had we oly measured the heights of the first te boys o this sample the value obtaied would have bee cm. If 2 boys had bee measured the the value would have bee cm. For a sample of 1 heights, the mea would have bee 18.1 cm. (this is a hypothetical sample geerated i the way described i Data Descriptio, Populatios ad the Normal Distributio assumig a populatio mea of 18 cm ad a populatio SD of 4.7 cm). These results ca be summarised i the followig table: Sample size Sample mea (cm.) Each of these sample meas is a legitimate estimate of μ - ideed, a sigle height measuremet, such as the first measuremet, cm, is a legitimate estimate of μ. Faced with this choice, which mea should be chose ad, more importatly, why? If issues of resources are igored (see later), the most ivestigators would ituitively say that the mea of the sample of size 1 was the best oe. This ituitio would be based o the otio that by usig data from 1 boys, the sample mea was based o more iformatio ad so must be better. A slightly more precise way of sayig this is to say that a large sample will provide a more represetative cross-sectio of the populatio ad therefore the mea of this sample would be closer to the mea of the whole populatio tha a mea based o a smaller sample. Priciples behid the Stadard Error The observatio that the mea of a large sample is, i some sese, goig to be closer to the mea of the populatio is the key to a more precise uderstadig of why the mea of a larger sample is better tha the mea of a smaller sample. I order to demostrate this, ad to formulate more precisely what is meat by better, it is useful to geerate ot just oe sample of data from a populatio, but may samples of the same size from that populatio. If we postulate particular parameter values (such as μ = 18 cm, σ=4.7 cm, as was doe above) the the computer ca geerate ay 1

2 umber of samples of ay give size. Of course, this is ot at all how thigs are i practice, where parameters are ukow. Moreover, drawig eve a sigle sample from a populatio ivolves a great deal of work ad drawig may samples is out of the questio. However, the followig is merely to demostrate the importat ideas: how the repeated drawig of samples is circumveted i practice is explaied the ext sectio. Suppose that 5 samples, each of size 1 are draw a Normal populatio with mea 18 ad stadard deviatio 4.7 (values that accord closely with those for the heights of five-year-old boys). The mea of each of these samples ca be computed ad a histogram of the resultig 5 meas ca be plotted: see figure 1. Samples of size Sample mea Figure 1 The pricipal thig to ote about figure 1 is that the histogram is much less dispersed tha the correspodig histogram of idividual heights see i figure 3 of Data Descriptio, Populatios ad the Normal Distributio. The data i the latter occupied the rage from 95 to 12 cm whereas the figure above is largely cofied to the iterval 15 to 112 cm. Two other features of figure 1 should also be oted: first the distributio of the sample meas does appear to be cetred o μ = 18 ad secod the distributio appears to have the shape of a Normal distributio. Repeatig this exercise but with 5 samples each of size 1 gives the histogram i figure 2. This histogram is also cetred o 18 cm but is eve less dispersed tha that i figure 1, with all the sample meas beig betwee 17 ad 19 cm ad virtually all of them beig betwee 17.5 ad 18.5 cm. 2

3 Samples of size fits2 Sample meas Figure 2 The differece betwee figures 1 ad 2 clearly shows the beefit of takig larger samples ad provides the basis for preferrig oe mea over aother. Meas based o larger samples provide more precise estimates of the uderlyig populatio mea because the distributio of the sample mea becomes more cocetrated about μ as the sample size icreases. Ideed, it is possible to be quatitative about this, as the stadard deviatios of the distributios i figures 1 ad 2 measure how precisely the sample mea estimates the populatio mea. The stadard deviatio of the meas of samples of size 1 (figure 1) is 1.54 cm ad the correspodig figure for samples of size 1 (figure 2) is cm. This figure measures how precisely the sample mea estimates the populatio mea ad is called the stadard error of the mea (SEM) or, more simply, the stadard error (SE). Calculatig the Stadard Error from a sigle sample Figures 1 ad 2 demostrate how the stadard error gives a useful measure of how far a sample estimate ca be expected to depart from the uderlyig parameter. However, i order to compute a value for the SE ot oe but may samples had to be draw from the populatio. This was achieved by askig the computer to geerate the samples. While this is a useful device for illustratig importat ideas, it is plaily artificial ad does ot provide a method that ca be applied i practice. The solutio to the problem is to use a theoretical result of great importace i statistics. This result states that: If a variable has populatio stadard deviatio σ the the stadard error of σ the mea of a sample of size is. 3

4 So, the SE of the mea of a sample of size 1 is σ/ 1.316σ, ad that of a sample of size 1 is te times less tha this, amely.316σ. For a stadard deviatio of 4.7 these figures are 1.49 ad 49 cm respectively. These are broadly similar to the values obtaied for the stadard deviatios of the distributios pictured i figures 1 ad 2, amely 1.54 ad cm respectively. σ The value of this formula is that we ca estimate the quatity from a sigle sample. The populatio stadard deviatio σ ca be estimated by the stadard deviatio, s, of the sigle sample that is to had, so the estimated SE is computed as s. Nomeclature It might be asked, whe the SE is simply the stadard deviatio of the distributio of sample meas, why a term other tha stadard deviatio is ecessary. I fact a reasoably coget argumet ca be made for abadoig the term stadard error. However, there are at least three reasos why stadard error is used. i) While the stadard error is a form of stadard deviatio, it is a very special form. It is the stadard deviatio of a hypothetical distributio that is ever actually observed. Values for the stadard error usually eed to be computed usig a theoretically derived formula. ii) iii) Stadard errors ad stadard deviatios are put to differet uses. Stadard deviatios are descriptive tools that idicate the dispersio i a sample. A stadard error is a iferetial tool, which measures the precisio of estimates of populatio parameters. The fact that a stadard error is a form of stadard deviatio ca readily give rise to cofusio. Stadard error is the term that has bee widely used for the stadard deviatio of the distributio of sample meas ad to chage omeclature ow may cause eve greater cofusio. Usig the Stadard Error The mea of the sample of 99 heights give i Data Descriptio, Populatios ad the Normal Distributio is cm ad its stadard error is.52 cm. While the foregoig discussio shows that the SE is a useful measure of how well the mea of this sample estimates the populatio mea μ, it is ot clear precisely how this iformatio should be used. It is commoplace to idicate the error i a quatity by quotig the value plus or mius the estimated error. Perhaps we could quote the mea as ±.52 cm.? Certaily, this is ofte the impressio give i the medical literature, where it is commo to fid table headigs of Mea (± SE) ad graphs with bars that are a stadard error i legth stretchig out above ad below some mea. However, this is a misleadig practice that should be discouraged. This is because it ivites the reader to suppose that the μ must lie betwee = ad = cm., which is false. 4

5 The problem ca be uderstood by recallig that for a Normal distributio most values (i fact about 95% of them) lie betwee μ 2σ ad μ + 2σ. As the distributio of sample meas has mea μ ad stadard deviatio equal to the SE, there is a 95% chace that the sample mea, m, is betwee μ ± 2SE, which amouts to sayig that there is a 95% chace that μ lies betwee m ± 2SE. Cosequetly it is much more accurate to assert that the populatio mea lies i the iterval m ± 2SE tha that implied by headigs such as mea ± SE. For the sample of heights of 99 boys, this iterval is (17.3, 19.38), which is wider tha the iterval i the previous paragraph. Cofidece Itervals σ σ For the reasos that have just bee outlied, the iterval m 2, m + 2 is, approximately, the 95% cofidece iterval for μ. A more exact defiitio is available ad is explaied i the Appedix. A alterative ame, widely used by methodological statisticias but ot ofte ecoutered i the applied literature, is iterval estimate (of μ). This is i distictio to the sigle-umber estimate m, which would be referred to as a poit estimate. This termiology reflects the fact that while a sigle value might have advatages as a estimate of a parameter, a sigle value caot adequately ackowledge the ucertaity i the estimate. For example, i a cliical trial of two agets iteded to reduce blood pressure, it would be difficult to iterpret the outcome that mea blood pressure o treatmet A is 1 mmhg lower tha o treatmet B. If a cofidece iterval o this differece was (-3, 5) mmhg the it could be reasoably cocluded that there was o material differece betwee the effect o blood pressure of A ad B. O the other had if the iterval was from (-3, 32) mmhg the there could well be a importat differece betwee the treatmets, eve though its precise ature has ot bee adequately elucidated by this trial. The differece betwee these alterative outcomes has oly become apparet through the use of cofidece itervals. It is for this reaso that may medical jourals ow isist o the use of cofidece itervals i the presetatio of results. Degree of Precisio It should be oted that the SE decreases as the sample size icreases, because the σ deomiator i the ratio gets larger. This is i distictio to the stadard deviatio, which does ot have a tedecy to get larger or smaller as icreases the sample stadard deviatio, s, simply becomes a better estimate of σ as icreases. However, the square root i the formula meas that the SE does ot decrease with sample size as quickly as might be hoped: i order to halve the SE the sample size must quadruple. It follows that a experimeter prepared to collect a sufficietly large sample could have a SE that was as small as they liked. However it is importat to esure that a This is more or less true, but a slight techicality has bee overlooked here 5

6 estimate has a precisio that is appropriate to the purpose of the ivestigatio, rather tha oe that is arbitrarily high. Collectig data o sufficiet patiets to determie the mea blood pressure of a group to withi 1 mmhg is likely to be a waste of time ad moey. Distributio of sample mea This is a useful place to explai a importat feature of the sample mea. Figures 1 ad 2 show that the meas of samples of Normally distributed variables do themselves have a Normal distributio. It is this fact that makes our defiitio of a cofidece iterval work: oly because the sample mea is Normally distributed ca we assert that 95% of sample meas are with 2 SEs of the populatio mea. However, eve whe the samples are of variables that are ot Normally distributed, the sample meas have a distributio that is ofte very close to a Normal distributio. This is a pheomeo explaied by the Cetral Limit Theorem (CLT) ad is illustrated i figure X (a) origial variable Sample mea Sample mea (b) meas of samples of size 1 (c) meas of samples of size 5 Figure 3 Figure 3 (a) is a histogram of 5 observatio from a populatio that has a skewed distributio. Figure 3 (b) is a histogram of the meas of 5 samples, each of size 1 ad figure 3 (c) is the correspodig histogram for samples of size 5. It is clear that the act of takig a mea of just te observatios has produced a quatity that has a much less skewed distributio ad takig the mea of 5 observatios gives a distributio that appears very close to Normal. A techical descriptio of why this is requires the reader to assimilate the details of the CLT. However, somethig of a heuristic explaatio is as follows. The mea of 6

7 the populatio show i figure 3 (a) is ad the skewess ca be thought of as arisig from the fact that a ecessarily positive the variable has substatial variatio. Cosequetly, large departures from mea with this value ca oly occur through values larger tha the mea, so a asymmetric distributio is ievitable. O the other had, sample meas will ted to be distributed close to the populatio mea (at least for sufficietly large samples), so they will deviate from the populatio mea by amouts that are small eough that the shape of the distributio is ot redered asymmetric by the fact that the sample meas must be positive. Appedix: exact form of the cofidece iterval (Not Assessed) The 95% cofidece iterval was itroduced as m ± 2 s /. While this versio of the cofidece iterval is adequate for may practical purposes it is, i fact, oly a approximatio to a more exact versio. A explaatio of why this is so is give below, so that the reader will ot be cofused by differeces betwee the above defiitio ad defiitios foud i textbooks. I additio, most statistics programs will use the exact method, so this will explai discrepacies betwee itervals calculated by widely used programs ad usig m ± 2 s /. There are two reasos why m ± 2 s / is oly approximate. The first is a mior poit ad stems from the fact that 95.45% of a Normal populat io lies betwee two stadard deviatios below ad two above the mea. Chagig the multiplier 2 to the somewhat less memorable 1.96 gives a iterval that ecloses 95% of the populatio. The secod poit is more subtle ad ivolves a poit which has bee glossed over i the above discussio. The SE is actually σ/ which, because it depeds o a parameter, σ, is ukow. The calculated cofidece iterval m ± 2 s / has substituted the sample stadard deviatio, s, for σ i the formula for the stadard error. However, just as m is ot a exact estimate of μ, either does s give the exact value for σ. Moreover, s teds vary about σ more i smaller tha i larger samples. A cosequece is that the iterval m ± 2 s / teds to be too arrow ad gives less tha 95% cofidece that it ecloses μ. The solutio is to replace the multiplier 2 by a larger value. However, the value chose will deped o the sample size, with larger values beig eeded for smaller samples. With large samples, typically i excess of 1, s will be a good estimate of σ ad the exact 95% cofidece iterval will be very close to m ± 1.96 s /. With smaller samples the discrepacy is greater. This is illustrated i the followig table, i which the approximate 95% cofidece iterval, m ± 2 s /, is compared with the exact versio, for both the full sample of 99 heights from Data Descriptio, Populatios ad the Normal Distributio ad for the sample of just the first te heights. Approximate Exact Sample of 1 heights (141, ) (13.74, 111.8) Sample of 99 heights (179, 19.39) (17.3,19.38) It ca be see that the exact iterval is oticeably wider for the smaller sample. For samples of size 99 the differece is uimportat (ad, to two decimal places, ot apparet). It is, i fact, the 97.5% poit of a Studet s t distributio with -1 degrees of freedom 7