5/10/2009. Significance Tests. Hypothesis Testing. Steps. How to choose for the alternative hypothesis? Step 1: The Claims

Transcription

1 Significance Tests Hypothesis Testing Chapter 8 Confidence intervals are one of the two most common types of statistical inference. Use a CI when your goal is to estimate a population parameter. The second type of inference, called tests of significance, has a different goal: test a claim concerning a population. 2 Steps Step 1: The Claims Claim 1 is The null hypothesis, H 0 : Always has = sign in it. nothing special is going on, no change from the status quo, no difference from the traditional state of affairs, no relationship. Claim 2 is The alternative hypothesis, H a : Has a <, or > or in it something is going on, there is a change from the status quo, there is a difference from the traditional state of affairs. The alternative hypothesis, Ha, usually represents what we want to check or what we suspect is really going on. 3 4 Step 1: Stating the hypotheses H 0 and H a Testing a population proportion H 0 : p = p 0 (where p 0 is a specific number, the claimed value). Testing a population mean H 0 : µ = µ 0 (where µ 0 is a specific number, the claimed value). How to choose for the alternative hypothesis? What determines the choice of a one-sided versus twosided test is the problem. You need to read the question! Ha : p < p 0 (one sided) Ha : p > p 0 (one sided) Ha : p p 0 (two sided) Ha : µ < µ 0 (one sided) Ha : µ > µ 0 (one sided) Ha : µ µ 0 (two sided) It is important to make that choice before performing the test or else you could make a choice of convenience or fall in circular logic

2 One-sided or two-sided? Ex.1: A health advocacy group tests whether the mean nicotine content of a brand of cigarettes is greater than the advertised value of 1.4 mg. This is a one-sided test: H 0 : µ = 1.4 mg H a : µ > 1.4 mg One-sided or two-sided? Ex.2: The FDA tests whether a generic drug has an absorption extent similar to the known absorption extent of the brand-name drug which is 8 hours. Higher or lower absorption would both be problematic, thus we test: H 0 : µ = 8 hours H a : µ 8 hours 7 8 One-sided or two-sided? Ex.3: More than 60% of all students at our college work part-time jobs during the academic year. H 0 : p = 0.60 H a : p > 0.60 Ex. 4: No more than one-third of cigarette smokers are interested in quitting. H 0 : p = 1/3 H a : p < 1/3 Step 2. Checking conditions/collecting data and summarizing them Testing a population proportion: Conditions SRS And np 0 > 10 n(1-p 0 ) > 10 Testing a population mean: Conditions SRS And either The population is normally distributed The sample size is 30 or greater 9 10 Step 2. Checking conditions/collecting data and summarizing them Testing a population proportion Since our parameter of interest is the population proportion p, once we collect the data, we find the sample proportion $p. Testing a population mean Since our parameter of interest is the population mean µ, once we collect the data, we find the sample mean x. Step 2. Checking conditions/collecting data and summarizing them Testing a population proportion Test statistic z = pˆ p 0 p 0 (1 p 0 ) n Testing a population mean Test statistic If σ is known: If σ is unknown: z = x µ σ n x µ t = s n

3 13 Step 3. Finding the p-value of the test The p-value -- the probability of getting data (test statistic) as extreme as that observed or even more extreme when Ho is true. For the z-test for the population mean is found exactly like the probability when we learned about probabilities for a normal distribution. You can use the z-table, or your calculator. 14 Upper-tail Ha : µ > µ 0 Ha : p > p 0 normalcdf(z-score, 99999) Lower-tail Ha : µ < µ 0 Ha : p < p 0 normalcdf(-99999, z-score) Third case: lower AND upper tail Ha : µ µ 0 Since the distribution is symmetric, find one of the tails and multiply by 2: 2 normalcdf( z, 99999) 2 normalcdf(-99999, - z ) Good news If you have a TI-83/84: STAT TESTS 1: Z-Test Use it when you test a population mean and you know σ 2: T-Test Use it when you test a population mean and you don t know σ 5: 1-PropZTest Use it when you test a population proportion Step 4. Conclusion You reach your conclusion using the p-value. p If the p-value > 0.10: little or no evidence against H < p-value 0.10: some evidence against H 0 (but not very strong) 0.01 < p-value 0.05: moderately strong evidence against H < p-value 0.01: strong evidence against H 0 p-value 0.001: very strong evidence against H 0 Also, we often compare the p-value with α, the significance level to state our conclusion. The most commonly used significance levels are 10%, 5%, and 1%. (Connection to the CIs!) Step 4. Conclusion If the p value < α, α then the data we got is considered to be rare (or surprising) enough when Ho is true, and we say that the data provide significant evidence against Ho, so we reject Ho and accept Ha. If the p value > α, α then our data are not considered to be surprising enough when Ho is true, and we say that our data do not provide enough evidence to reject Ho (or equivalently, that the data does not provide enough evidence to accept Ha). Comment about wording: Another common wording is: "The results are statistically significant" - when p value < α "The results are not statistically significant" - when p value > α

4 Step 4. Conclusion p-value > α Do not reject the null hypothesis p-value < α Reject the null hypothesis Acceptance level/significance level, α Confidence intervals to test hypotheses Because a two-sided test is symmetrical, you can also use a confidence interval to test a two-sided hypothesis. ONLY for two-sided tests! For a significance level α, you can use a 1 α CI to test the hypothesis. If the claimed value is inside the CI, it is a plausible value do not reject the null hypothesis. If the claimed value is outside of the CI, it is NOT a plausible value reject the null hypothesis Logic of confidence interval test Ex: Your sample gives a 99% confidence interval of x ± E = 084. ± = ( 0829., ) With 99% confidence, could samples be from populations with µ =0.86? µ =0.848? At the 1% level, cannot reject H 0 : µ = At the 1% level, 99% C.I. reject H 0 : µ = 0.86 The only way to be absolutely certain of whether H 0 is true or false is to test the entire population. Because your decision is based on a sample, you must accept the fact that your decision might be incorrect. You might reject a null hypothesis when it is actually true, or you might fail to reject a null hypothesis when it is actually false. x Think of a household smoke-detector. If it goes off every time you make the toast too dark we have an alarm without a fire. We call this Type I error. 23 Conversely, a Type II error is a fire without alarm. Now think of the null hypothesis as the condition of NO FIRE, while the alternative hypothesis is that a FIRE is burning. 24 The goal is to avoid these errors. Every cook knows how to avoid a Type I error: just remove the batteries! But this increases the incidence of Type II errors. Similarly, reducing the chances of Type II error, for example by making the alarm hypersensitive, can increase the number of false alarms. No Fire Fire No alarm No error Type II error Alarm Type I error No error 4

5 Decision: Not guilty Defendant: Not guilty No error Defendant: guilty Type II error Decision: Guilty Type I error No error H 0 is true H a is true Do not reject H 0 No error Type II error Reject H 0 Type I error No error Important comments We test the null hypothesis assuming it is true. We never ever make claims about the sample. We KNOW everything about the sample we don t need to test it. Conclusion: We either reject the null hypothesis or we do not reject the null hypothesis. The p-value summarizes the evidence by describing how unusual the data would be if H 0 were true To convince ourselves that H a is true, we must show that data contradict H 0 If the p-value is small, the data contradict H 0 and support H a A p-value is a probability. Its value must be between 0 and 1. A p-value is NOT the probability that the null hypothesis is true A consumer advocate is interested in evaluating the claim that a new granola cereal contains 4 ounces of cashews in every bag. The advocate recognizes that the amount of cashews will vary slightly from bag to bag, but she suspects that the mean amount of cashews per bag is less than 4 ounces. To check the claim, the advocate purchases a random sample of 40 bags of cereal and calculates a sample mean of 3.68 ounces of cashews. The consumer advocate should declare statistical significance only if there is a small probability of a) Observing a sample mean of 3.68 oz. or less when µ = 4 oz. b) Observing a sample mean of exactly 3.68 oz. when µ = 4 oz. c) Observing a sample mean of 3.68 oz. or greater when µ = 4 oz. d) Observing a sample mean of less than 4 oz. when µ = 4 oz. 27 Stating hypotheses A consumer advocate is interested in evaluating the claim that a new granola cereal contains 4 ounces of cashews in every bag. The advocate recognizes that the amount of cashews will vary slightly from bag to bag, but she suspects that the mean amount of cashews per bag is less than 4 ounces. To check the claim, the advocate purchases a random sample of 40 bags of cereal and calculates a sample mean of 3.68 ounces of cashews. What alternative hypothesis does she want to test? a) b) c) d) e) f) 28 A consumer advocate is interested in evaluating the claim that a new granola cereal contains 4 ounces of cashews in every bag. The advocate recognizes that the amount of cashews will vary slightly from bag to bag, but she suspects that the mean amount of cashews per bag is less than 4 ounces. To check the claim, the advocate purchases a random sample of 40 bags of cereal and calculates a sample mean of 3.68 ounces of cashews. Suppose the consumer advocate computes the probability described in the previous question to be Her result is a) Statistically significant. b) Not statistically significant A consumer advocate is interested in evaluating the claim that a new granola cereal contains 4 ounces of cashews in every bag. The advocate recognizes that the amount of cashews will vary slightly from bag to bag, but she suspects that the mean amount of cashews per bag is less than 4 ounces. To check the claim, the advocate purchases a random sample of 40 bags of cereal and calculates a sample mean of 3.68 ounces of cashews. If the probability described in the previous question is , the difference between the observed value and the hypothesized value is a) Small. b) Large. 5

6 A consumer advocate is interested in evaluating the claim that a new granola cereal contains 4 ounces of cashews in every bag. The advocate recognizes that the amount of cashews will vary slightly from bag to bag, but she suspects that the mean amount of cashews per bag is less than 4 ounces. To check the claim, the advocate purchases a random sample of 40 bags of cereal and calculates a sample mean of 3.68 ounces of cashews. If the probability described in the previous question is , then the consumer advocate should conclude that the granola is a) Correctly labeled. b) Incorrectly labeled We reject the null hypothesis whenever a) P-value > α. b) P-value α. c) P-value α. d) P-value µ. The significance level is denoted by a) µ b) σ c) α d) P-value Which of the following is the most common choice for significance level? a) 0 b) 0.01 c) 0.25 d) 0.50 e) Calculating P-values To calculate the P-value for a significance test, we need to use information about the Conclusions Suppose the P-value for a hypothesis test is Using α = 0.05, what is the appropriate conclusion? a) Sample distribution b) Population distribution c) Sampling distribution of a) Reject the null hypothesis. b) Reject the alternative hypothesis. c) Do not reject the null hypothesis. d) Do not reject the alternative hypothesis

7 Conclusions Suppose the P-value for a hypothesis test is Using α = 0.05, what is the appropriate conclusion? a) Reject the null hypothesis. b) Reject the alternative hypothesis. c) Do not reject the null hypothesis. d) Do not reject the alternative hypothesis. P-value True or False: The P-value should be calculated BEFE choosing the significance level for the test. a) True b) False Conclusions Suppose a significance test is being conducted using a significance level of If a student calculates a P- value of 1.9, the student a) Should reject the null hypothesis. b) Should fail to reject the null hypothesis. c) Made a mistake in calculating the P-value. Stating hypotheses If we test H 0 : µ = 40 vs. H a : µ < 40, this test is a) One-sided (left tail). b) One-sided (right tail). c) Two-sided Stating hypotheses If we test H 0 : µ = 40 vs. H a : µ 40, this test is a) One-sided (left tail). b) One-sided (right tail). c) Two-sided. Using CIs to test A researcher is interested in estimating the mean yield (in bushels per acre) of a variety of corn. From her sample, she calculates the following 95% confidence interval: (118.74, ). Her colleague wants to test (at α = 0.05) whether or not the mean yield for the population is different from 120 bushels per acre. Based on the given confidence interval, what can the colleague conclude? a) The mean yield is different from 120 and it is statistically significant. b) The mean yield is not different from 120 and it is statistically significant. c) The mean yield is different from 120 and it is not statistically significant. d) The mean yield is not different from 120 and it is not statistically significant. 42 7