Summary of method. We assume we have a sample mean x, with a standard deviation s, calculated from a random sample of size n.

Transcription

1 Summary of method The method is designed to test whether the mean of some population, µ say, is equal to some predetermined value µ 0, where µ 0 could be the result for a larger population, or from an older survey, or could be some target value (e.g. specifying that the mean concentration of an environmental pollutant should be lower than a safe level determined by the EPA). We assume we have a sample mean x, with a standard deviation s, calculated from a random sample of size n. 1

2 1. Determine H 0 : µ = µ 0 and H a (either µ > µ 0 or µ < µ 0 or µ µ 0 ) 2. Calculate the test statistic 3. Calculate df = n 1. t = x µ 0 S.E. = x µ 0 s/ n. 4. Determine the P value either exactly using Excel or Minitab, or using Table B in the back of the book. Note that if you use Table B,, you will not be able to determine P exactly but will be able to identify the nearest value among 0.1, 0.05, 0.025, 0.01, and (one-sided) or among 0.2, 0.1, 0.05, 0.02, 0.01 and (two-sided), 5. Report your conclusions. Usually if P> 0.05 we simply report that the result was not statistically significant, but if P 0.05 we report that fact and our nearest guess for the exact value of P. 2

3 Example Problems Question 8.69, page 420. A bank wants to evaluate which of two credit cards is more attractive to its customers. For a random sample of 100 customers, 40 say they would prefer one that has an annual cost but low interest. Software reports: Test of p=0.50 vs p not = 0.50 X N Sample p 95& CI Z-value P-value (0.304,0.496) Explain how to interpret these results. What would you report to the company? 3

4 Solution. For calculating a confidence interval, the sample proportion is ˆp and the standard error is ˆp(1 ˆp) n = =.049. Hence the 95% CI is 0.4 ± = 0.04 ± = (0.304, 0.496). For the hypothesis test of H 0 : p = p 0 = 0.5, we recalculate the standard error based on p 0, as = Then the z statistic is = 2.0, and the corresponding one-sided P- value is.0228 (from Table A). Therefore, the two-sided P-value is = [Slight rounding error here: the exact one-sided P-value is and twice that would be.0455, but this makes no difference in practice.] Report to the company: Less than half the customers favor the new card, and that is a statistically significant difference (though only barely). 4

5 Question 8.73, page 420. For a quantitative variable you want to test H 0 : µ = 0 against H a : µ 0. The 10 observations are 3,7,3,3,0,8,1,12,5,8. 1. Show: (a) x = 5.0, (b) s = 3.71, (c) standard error is 1.17, (d) test statistic is 4.26, (e) df=9. 2. The P-value is Interpret, and make a decision using a significance level of If you had instead used H a : µ > 0, what would the P-value be? Interpret. 4. If you had instead used H a : µ < 0, what would the P-value be? Interpret. 5

6 Solution 1. (a) and (b) are direct calculation (you could also use Excel). (c) 3.71 = (d) t = 5 0 = 4.27 (slight rounding error) (e) df = n 1 = 10 1 = Since < 0.05, we reject the null hypothesis. Based on the data, there seems to be significant evidence that the mean is not For a one-sided test, the P-value is half that for a two-sided test, i.e This also leads to the conclusion that the mean is not The P-value for H a : µ < 0 would be the probability that t < 4.27 based on df = 9. This is 1 minus the probability that t > 4.27, i.e. about In this case, since the only alternative of interest is µ < 0 and the data support µ > 0, we have no choice but to accept H 0. 6

7 Limitations of Significance Tests 1. Statistical significance does not necessarily mean practical significance. 2. Significance tests are generally less useful than confidence intervals. 7

8 Example. Suppose we take a group of 200 patients whose mean cholesterol level is 250 or higher. We given them a cholesterolreducing drug and measure the reduction in cholesterol after some period of time. Over all the patients, the mean reduction in cholesterol is 10 points and the standard deviation is 50. Is this a significant result? 8

9 The standard error is 50 = 3.53 so we calculate z = = The associated P-value is approximately (one-sided). So it is (statistically) significant. But is it practically significant? Most doctors recommend a cholesterol level lower than 200 a reduction of 10 points is not necessarily of practical significance for someone over 250. In other words, the difference is big enough that it could not have occured by chance (that s the definition of statistical significance), but that doesn t mean the drug is effective in practice. 9

10 The alternative approach is to calculate a confidence interval for the mean reduction in cholesterol. Since x = 10 and the standard error is 3.53, a 95% confidence interval is 10 ± = (3.1, 16.9). In the opinion of most statisticians, that gives a much clearer picture of the uncertainty in the data than simply saying we rejected the null hypothesis µ = 0. 10

11 Common Misinterpretations of Significance Tests 1. If a significance test results in the conclusion do not reject H 0, that does not mean H 0 is true. 2. The P-value is not the probability that H 0 is true. 3. Many medical researchers only report their results if they are statistically significant. This leads to publication bias. 4. Some results are statistically significant by chance. Therefore, you cannot conclude that every significant result published in the literature is true. 11

12 Type I and Type II Errors Type I Error: H 0 true but rejected Type II Error: H 0 false but not rejected The significance level is the probability of a type I error. 12

13 Example A new drug therapy is proposed to reduce the risk of heart attack. Among the category of patients for whom the drug is intended, the chance of a heart attack within 5 years is considered to be 25%. We test the drug on 120 patients so that the type I error is 0.01 in a one-sided test (H 0 : p = 0.25 against H a : p < 0.25). Suppose the drug is effective, so that for a patient who takes the drug, the chance of a heart attack within 5 years is only 10%. What is the probability of a type II error? 13

14 First define the test for a significance level of The standard error is = Reject H 0 if ˆp < = (The z value 2.33 comes from the 0.01 significance level, via Table A.) If p = 0.1, the standard error is now = The probability that ˆp < is derived from z = = The probability associated with that value of z is The probability of type II error is =

15 Example: Question 8.57, page 416. For the example of an astrologer trying to identify someone s personality profile, the probability that the astrologer is correct if she is randomly guessing is 1 3. Suppose we test H 0 : p = 1 3 against H a : p 3 1 when the sample size is n = 116 and the significance level is Suppose also the true value of p is Show that a type II error occurs if < ˆp < Show that when p = 1 2, the probability that ˆp < is 0 and the probability that ˆp > is Show that the probability of a type II error is

16 Solution 1. For null hypothesis p 0 = 1 3, the standard error is p 0 (1 p 0 ) 116 =.0438 so we reject H 0 if ˆp < = or if ˆp > = (you may get slightly different answers because of rounding error) 2. If p = 0.5, the standard deviation of ˆp is = For the probability that ˆp < x when x =.248, we calculate z = = 5.43, for which the associated left-tail probability is 0.00 to 2 decimal places. For x =.419, compute z = = 1.75, for which the left-tail probability is Hence the right-tail probability is 0.96, as required. 3. If the true p is 0.5, then the probability that ˆp is between and is = Hence this is the probability of a type II error. 16