Inference, Sampling, and Confidence. Inference. Inference

Transcription

1 Inference, Sampling, and Confidence Inference defined Sampling Statistics and parameters Sampling distribution Confidence and standard error Estimation Precision and accuracy Estimating sample size 1 Inference Sample Population Observations Statistics Inference (aka Estimation) 2 Inference the act or process of inferring: the act of passing from statistical sample data to generalizations (as of the value of population parameters) usually with calculated degrees of certainty Merriam-Webster 1998 The strength (certainty) of your inference is largely dependent upon your sampling (both methodology and intensity). But, the sample must be representative of the population. 3

2 Representative Sampling Consider a maple tree from which you wished to describe mean leaf length from. Where would you sample from? Lower leaves = shade leaves Upper Leaves = sun leaves Avoid BIASED sampling! No biological validity if biased! 4 Obtaining a Representative Sample RANDOMNESS Insure that every member of the population has an equal probability of being sampled. CAUTION!!! A random sample can still be biased! Randomness only refers to how the observations were selected for the sample. No guarantee that sample is representative. (Chance alone may affect outcome.) 5 Obtaining a Representative Sample STRATIFICATION If there is obvious bimodality, stratify the sample (split) and randomly sample WITHIN each stratified segment. Example of Stratification: This is why we sample the tree stratum and seedling stratum separately in vegetation studies. 6

3 Random Haphazard! Random An element of a set whose members all have an equal probability of occurrence. Haphazard (Sample of Convenience) By mere chance, accident, or fortuity; without design, casually. (Probabilities unequal.) The Oxford English Dictionary, 2nd ed. 7 Statistics and Parameters Values used to describe populations are referred to as parameters (Greek) (e.g., standard deviation = σ) Values used to describe samples are referred to as statistics (Roman) (e.g., standard deviation = S) 8 Parameters and Statistics Population μ ± σ Parameters Y isan estimate of Sample Y ± S Statistics 9

4 Statistics and Parameters Y is said to be an unbiased estimate of μ because if we were to draw an infinite number of samples of size N, with replacement, the mean of those samples would equal μ. However, the mean of all the variances S 2 would NOT equal σ 2. The former would be smaller. Thus, we say S 2 is a biased estimate of σ Why is S 2 a Biased Estimate of σ 2? Recall the maple leaf example: If we made 100 samples of N = 10 leaves each, it is very improbable that we would ever sample the full range of leaf sizes. We would likely always miss the smallest and largest values from the population. We are unable to ever sample the full range (variance) of the population. 11 Can We Reduce This Bias? Yes. Recall our general theoretical formula for the variance: Subtracting 1 from the denominator increases the value of S 2 and more closely approximates σ 2 : S 2 = N i=1 Y i Y 2 N S 2 = N i=1 Y i Y 2 N 1 See any text in theoretical statistics for a mathematical proof. 12

5 Sampling Distributions Note that we can use Y and S 2 as estimators of μ and σ 2 if the target population is normal, regardless of the sample size. The Central Limit Theorem suggests that as sample sizes (N) become large, the shape of most samples begins to approximate a normal distribution. Note that in addition to the normal distribution, there are many other distributions that can be used for statistical inference (t-, F-, χ 2, etc.). 13 Central Limit Theorem 14 Confidence Intervals and Standard Errors Earlier, we introduced the notion of strength or certainty of inference. Formally, this statistical notion is referred to as confidence. In any given experimental situation we can calculate our statistical confidence that our sample statistics accurately reflect our population parameters. In other words, what is the probability that we have made a mistake? 15

6 Confidence 1 Suppose we know μ of a population. Suppose also that we collected 20 samples (N = 10 each) from that pop in an effort to describe μ. μ If 1 of the 20 samples was unable to describe the mean (one sample did not contain μ), we can say we have 95% confidence in our ability to sample the population and accurately describe its mean Standard Error of the Mean From the previous illustration we can see that the mean of all the means should closely approximate μ. We can also approximate the standard deviation of a sampling distribution of means. This is referred to as the Standard Error (SE): Y N Approx. as: SE S N 18

7 Estimation Now, suppose a sample is drawn from a population with unknown μ and σ : Sample: N = 100,Y = 50, S = 5 Assuming an infinite number of samples of N = 100, we can calculate SE = S/ N = 5/10 = 0.5 Q: What does a SE of 0.5 represent? A: A standard deviation of our sampled means. ± 1 SE would represent 68% of the area (± 2 SE ca. 95%) In other words, 68% of the time the mean would fall between 49.5 and 50.5 (we have 68% confidence). 19 How Much Confidence is Enough? From the preceding example, we had 68% confidence in our prediction of the mean. Unfortunately, to be looked upon with favor by our colleagues we would need at least 95% confidence. Q: How do we get 95%? A: Use Z-values and our knowledge of the SND. With 95% of the area, we know there is 5% remaining (2.5% in each tail). So, in Appendix B we look up (0.05/2 = 0.025) and find a Z- value of How Much Confidence is Enough? We can now use our determined Z-value of 1.96 to calculate the 95% Confidence Interval around the mean (the 2SE Rule of Thumb): CI 0.95 : Y ±1.96 S N In our example: CI 0.95 : 50 ± 1.96 (0.50) CI 0.95 : NB: A CI can be calculated for any statistic! 21

8 Precision & Accuracy Re-Visited Remember our discussion of precision & accuracy? Because SE = S/ N, the greater N becomes, the smaller SE becomes. So, as N increases, SE decreases (the precision of our estimate of μ increases)! This is why we normally wish N to be as large as is feasible for a given experiment. (Within TME limits.) Precision: depends upon sample size (CI) Accuracy: depends on sampling protocol 22 Sample Size Q: So, if sample size is so important, how large of a sample is needed to describe a population? A: It depends McCarthy s Quick Guidelines: N Adequacy 1 Poor 2-4 Fair 5-10 Good Better >15 Best 23 Sample Size General guidelines are useful for general practice but sometimes an explicit estimate of precision is required. Index of Precision: D = SE / Y For example, given SE = 2 and Y = 20 The level of precision is 10% of the mean (D = 2/20 = 0.10) 24

9 Index of Relative Precision D would be better expressed as a CI 0.95 : D'=t 0.05 /2,df SE Y 100 Thus, you are expressing precision as a percentage of the sample mean at the traditional 95% CI. D is more straightforward to conceptualize than D. 25 Calculating Relative Precision - Example - Sample: 5, 3, 4, 4 N = 4 Y = 4 S = SE = (S/ N) Conclusion: More sampling is required D = (0.408 / 4) * 100 = 32.5% Note 1 : df = N - 1, t 0.025,3 = 3.18 (Appendix C) Note 2 : 32.5% is high; < 20% preferred, < 10% better 26 Estimating Desired Sample Size - Using Pilot Data - If a small, preliminary (pilot) study provides a fair description of the mean and variance, then you can estimate N needed for a desired level of precision: ^ N = desired N t = t-value from App. C (95CI) at df from pilot data S = std. dev. of pilot data D = D expressed in decimal form (not %!) Y = mean of pilot data N = t /2,df S D'' Y *Caveat: final sample will have different mean and variance! 27

10 Estimating Desired Sample Size - Example - Assume a pilot study with N = 10,Y = 4, S = 2. Assume also the desired level of precision to be D = 20% (i.e., a 95% CI ± 20% of Y). Thus, D = 0.20 and t 0.05/2,9 = N = = Conclusion: A sample size of 32 is needed to get this level of precision 28