Questions about the Assignment For Part II, only one person calculated the 95% confidence interval using the standard error method. Exam #1 Wednesday, July 18 th Exam will be written and in-class. They will be a combination of defining terms, solving problems, and interpreting/discussing results. Exams will be closed book and closed computer. You may bring a (non-cell phone) calculator and one double-sided 8 ½ x 11 page of notes. You must prepare this page of notes yourself and submit it along with your exam. There will be no make-up for exams. If an exam must be missed, absence must be officially excused in advance. Hypothesis Testing II What is the probability that our observed outcome could have occurred by random chance? Hypothesis Testing II andomization distribution Statistical significance The is the probability of getting a sample statistic as extreme as the observed sample statistic, just by random chance, if the null hypothesis is true. The smaller the (i.e., the smaller the probability), the stronger the evidence is against the H 0 and in favor of the H a. ight Tail Enter 3 Exercise and Gender Study We use the randomization sampling distribution to calculate the of the observed sample statistic. The is the proportion of randomization sample statistics that are as extreme as our observed sample statistic. You could get the p- value by counting the red dots. 1
Exercise and Gender Study If time spent exercising did not differ by gender, we would see a difference in sample means as extreme as 3 hours in about 10% of our studies. Example: The observed sample statistic from Study A has a of 0.002 and the observed sample statistic from Study B has a of 0.2. Which study provides stronger evidence against the null hypothesis? The lower the, the stronger the evidence against the null hypothesis. A. Study A B. Study B C. Study A and Study B provide equally strong evidence ight Tail Enter 3 An Experiment on esearch Question: Is effective at treating cocaine In a randomized experiment on treating cocaine addiction 48 cocaine addicts were randomly assigned to take either (a new drug) or a placebo. Then they were followed to see who relapsed. Sample size (n) = 48 cocaine addicts Two Variables: Treatment given: or a placebo Outcome: elapsed or No elapse What are the sample statistics we need for this study? D: The proportion of people treated with who relapsed P: The proportion of people treated with a placebo who relapsed esearch Question: Is effective at treating cocaine The null hypothesis is the claim that there is no effect or no difference What would be our null hypothesis for this experiment? is equally effective as a placebo at treating cocaine addiction. H 0 : p D = p P (or p D p P = 0) The alternative hypothesis is the claim that we seek evidence for. What would be our alternative hypothesis? is more effective than a placebo at treating cocaine addiction. H a : p D < p P (or p D p P < 0) Conducting the Experiment Conducting the Experiment 1. andomly assign participants to treatment groups 2. Carrying out the treatment phase for both groups 3. Observe relapse counts in each group = elapsed N = No elapse P P P P 1. andomly assign participants to treatment groups P P P P 24 Participants 24 Participants N N Observed Sample Statistic = P = =.416 10 relapsed, 14 no relapse 20 relapsed, 4 no relapse 2
Measuring Evidence against H 0 esearch Question: Is effective at treating cocaine Two options: 1. H 0 is true ( and the placebo cause the same proportion of relapses) To see if an observed sample statistic provides evidence against H 0, we need to see what kind of sample statistics we would observe, just by random chance, if H 0 was true. 2. H a is true ( causes a smaller proportion of relapses than a placebo) If H 0 is true, how would you explain the observed difference in the sample proportion of relapses? The observed difference in the sample proportion of relapses could have reasonably happen by chance. How can we determine the probability that our observed sample statistic could have occurred by random chance? Statistical Test esearch Question: Is effective at treating cocaine Observed sample statistic: P = -.416 (Diff. in sample proportions) How unusual would it be to observe this sample statistic by random chance if the null hypothesis was true (i.e., P = 0)? What is the probability that we would observe, by random chance, a difference in sample proportions as large as.416 if is equally effective as a placebo at treating cocaine To answer this question we need a distribution of sample statistics that would occur if the null hypothesis was true. andomization Process The sample size of our observed sample is 48. Imagine having 48 pieces of paper. 30 pieces have an on it and 18 have an N on it. This corresponds with the total number of elapsers and Nonrelapsers in our experiment (i.e., observed sample). We want to generate samples where the null hypothesis is true (i.e., is equally effective as a placebo at treating cocaine addiction). To do this we can randomly assign each piece of paper to a treatment group. To be consistent with our observed sample, we d randomly assign 24 pieces to the group and 24 pieces to the placebo group. Then we would calculate the sample statistic (i.e., difference in sample proportions) for this randomization sample. We can generate these sample statistics using the randomization process. Create a andomization Sample Create a andomization Sample N N N N N N Our Observed Sample N N N N N N 10 relapsed, 14 no relapse 20 relapsed, 4 no relapse N N N N N N N andomization Sample Statistic = P = =.084 N N N N N N N N N 16 relapsed, 8 no relapse 14 relapsed, 10 no relapse 3
Create a andomization Sampling Distribution epeat this process 1,000 times to obtain 1,000 randomization sample statistics to form a randomization sampling distribution. N N N N N andomization Sample Statistic = P = N N N N N N N N N Proportion of randomization sample statistics as extreme as the observed sample statistic of the observed sample statistic The probability of getting a sample difference in proportions as low as -0.416 just by random chance, if is equally effective as a placebo, is 0.003 N N =.166 N N 17 relapsed, 7 no relapse 13 relapsed, 11 no relapse The observed sample statistic Alternative Hypothesis Proportion of randomization sample statistics as extreme as the observed sample statistic The observed sample statistic of the observed sample statistic The is the area in the tail(s) beyond the observed sample statistic in the randomization sampling distribution. Which tail(s) to include (i.e., lefttail, right-tail, or two-tail ) depends on the alternative hypothesis. The alternative hypothesis is determined by the research question. A one-sided H a contains either > or < A two-sided H a contains For a one-sided H a, the is the proportion of randomization sample statistics in the tail specified by H a (i.e., < left-tail and > right-tail). For a two-sided H a, the is the proportion of randomization sample statistics in both tails. Exercise and Gender A Two-Tail Test esearch Question: Among college students, does one gender spend more time exercising than the other? What are the parameters of interest? = mean number of hours male students spend exercising = mean number of hours female students spend exercising What is the H 0 and H a? H 0 : - 0 Time spent exercising does not differ by gender. H a : - 0 Time spent exercising does differ by gender. Exercise and Gender www.lock5stat.com/statkey = 2 x.109 = 0.218 Little evidence against H 0 Do not reject H 0 Conclusion: This study does not provide adequate evidence that there is any association between gender and exercise times among college students. Think: A result this extreme would happen about 22% of the time just by random chance if H 0 were true, so this study does not provide adequate evidence against H 0. 4
Strength of Evidence The is the probability of getting results as extreme as our observed sample statistic, if the null hypothesis is true. The measures our evidence against the null hypothesis. Hypothesis Testing If the is small enough, we reject the null hypothesis, in favor of the alternative hypothesis How small is small enough?.01.05.10 s 1 The smaller the, the smaller the proportion of randomization sample statistics as extreme as our sample statistic. The smaller the, the stronger the evidence against H 0. Statistical Significance The significance level ( ) is the threshold (e.g.,.05,.01) below which the is deemed small enough to reject the null hypothesis. If the is less than the threshold, the results are statistically significant, and we reject the null hypothesis in favor of the alternative hypothesis. When the proportion of randomization sample statistics as extreme as our observed sample statistic is less than (e.g.,.05,.01), we say that our observed sample statistic is statistically significant. Saying that our observed sample statistic is statistically significant, means that we have convincing evidence against H 0 (and for H a ) Statistical Conclusions Strength of evidence against H 0 :.01.05.10 s Formal decision of hypothesis test [based on = 0.05]:.01.05.10 statistically significant [ ] not statistically significant [ ] 1 1 Formal Decisions A formal hypothesis test has only two possible conclusions: 1. If the is : eject the null hypothesis in favor of the alternative. Part I: 4.52, 4.76, and 4.84 Hint for #4.84: A correlation (r) between two variables is a type of sample statistic Part II: See Next Slide Assignment 2. If the is : Do not reject the null hypothesis. 5
Assignment Obtaining Proportions from the GSS Part II: (Type up this assignment in a Word document) [Worth 100 points] Construct a research question that uses the following GSS variables DIVOCE and SEX. Provide the symbol and value for the sample mean/sample proportion for each variable. Provide the symbol and value for the sample statistic you ll be testing. State your null hypothesis in words and with an equation. State your alternative hypothesis in words and with an equation. Indicate whether this will be a left-tail, right-tail or two-tail test. Use StatKey to generate a randomization sampling distribution where the H 0 is true. (Provide a screen shot of your randomization sampling distribution) Calculate and interpret the for your observed sample statistic. Assess the strength of evidence this data provides against H 0 Select a significance level and make a formal decision based on the significance level Interpret/explain the results/conclusions of your study. Hint: This is similar to the Cocaine study (i.e., difference in proportions) To compare proportions across two variables. Enter the first variable here and the second variable here. Uncheck the Weighted box Check the Unweighted box Click this button and the values/statistics needed to calculate the difference in sample proportions will open up in a new window. Entering Data into StatKey to Create a andomization Sampling Distribution Identifying where your Observed Sample Statistic fits on the andomization Sampling Distribution Click this button to enter your data and this window will pop up. Once you ve created your randomization sampling distribution. Check the appropriate tail test box. To see where your observed sample statistic fits on this distribution by click here. This window will pop up and you can enter the value for your observed sample statistic here. Summary A randomization sampling distribution shows the distribution of statistics that would be observed if H 0 was true. A is the probability of getting a sample statistic as extreme as the observed sample statistic, just by random chance, if H 0 is true. The measures the strength of evidence against H 0. esults are statistically significant if the is < α (the significance level). In making formal decisions, reject H 0 if the < α; otherwise do not reject H 0. Hypothesis Testing 1. Construct research question 2. Define the parameter(s) of interest 3. State H 0 and H a 4. Set significance level ( ) [usually 0.05 if unspecified] 5. Collect data 6. Generate descriptive statistics 7. Calculate the appropriate observed sample statistic 8. Create a randomization sampling distribution (where H 0 is true) 9. Calculate the of the observed sample statistic 10.Assess the strength of evidence against H 0 11.Make a formal decision based on the significance level 12.Interpret the conclusion in context 6
andomized Experiments In randomized experiments the randomness is the random allocation of cases to treatment groups. If the null hypothesis is true, it doesn t make any difference which treatment group a respondent gets placed in. Generate randomization samples assuming H 0 is true by reallocating units to treatment groups, and keeping the response values the same. Formal Decisions eject H 0 if observing a sample statistic so extreme is unlikely when H 0 is true. This means that the observed sample data provides strong evidence to support H a. Do not reject H 0 if observing a sample statistic is likely when H 0 is true. This means that the observed sample data does not provide strong enough evidence to reject H 0 (and support H a ) For a given significance level ( ) < eject H 0 > Do not eject H 0 Elephant Example The mystery animal X is unknown, so we set up the following hypothesis test: H 0 : X is an elephant H a : X is not an elephant What would you conclude, if you had the following data? X has four legs Since it remains plausible that X could be an elephant, we don t have enough evidence to reject H 0. However, with this data we also cannot accept H 0 and conclude that X is an elephant. X walks on two legs Since it is highly unusual for an elephant to walk on two legs, we can reject H 0 and conclude that X is probably not an elephant. andomization Process Through the randomization process we can generate a randomization sampling distribution which is the distribution of sample statistics we would observe, just by random chance, if the null hypothesis was true. 1. Simulate many randomization samples, assuming H 0 is true. 2. For each randomization sample, calculate the randomization sample statistic. 3. These randomization sample statistics form a randomization sampling distribution. 4. Find the proportion of these randomization sample statistics that are as extreme as our observed sample statistic. Statistical Significance www.xkcd.com 7