Business Statistics. Lecture 8: More Hypothesis Testing

Transcription

1 Business Statistics Lecture 8: More Hypothesis Testing 1

2 Goals for this Lecture Review of t-tests Additional hypothesis tests Two-sample tests Paired tests 2

3 The Basic Idea of Hypothesis Testing Start with a theory or hypothesis For example, m = Collect some data Ask: How unusual is it to see this data if the null hypothesis is true? If it s unusual, reject the null hypothesis If not, fail to reject the null Remember, determine the hypothesis to be tested before looking before looking at the data 3

4 It All Ties Back to the Empirical Rule 68% 95% Z If we hypothesize that the data come from a N(0,1) distribution, how unusual an observation must we see to reject our hypothesis? It depends on the alternative hypothesis 4

5 For Example, a Two-sided Test Null: The mean is equal to zero (H 0 : m = 0) Alternative: The mean is not equal to zero (H a : m 0) If the rejection criterion is p-value < 0.05, we reject if our observation is greater than 1.96 or less than -1.96: 68% 95% Z 5

6 In JMP JMP computes the probability of seeing data as extreme or more extreme under various alternate hypotheses You have to choose the appropriate p-value Then compare the JMP p-value to 0.05 Smaller: reject the null Larger: fail to reject the null Output is in terms of rescaled t-scores Using t distribution comes from using s to estimate s 6

7 Conducting the Test in JMP With one continuous variable, Analyze > Distribution > red triangle > Test Mean Type in the mean to be tested ( Specify Hypothesized Mean ) If population ( true ) standard deviation known, enter it This will be a z-test If you leave it blank, JMP does a t-test It uses s to estimate s 7

8 Back to the Paint Case (primer.jmp) A More Complicated Question: Suppose we are less interested in the value of 1.2 and more interested in whether processes a and b have the same mean Null hypothesis Means are the same: m a - m b = 0 Alternative hypothesis Means are different: m a - m b 0 8

9 Solution: Two-sample t-test Process a Mean = m x X SD = s x X 1, X 2,, X n Random Samples Process b Mean = m y SD = s y Y 1, Y 2,, Y m Y Two sample t-test assumes Xs and Ys are independent 9

10 Results of Two Sample t-test What do you think the test statistic is? How should we rescale the test statistic? What does the p-value represent? 10

11 Null Hypothesis: m x - m y = 0 Test Statistic: Fact: since X and Y are independent: So Var Two-sample t-test X Y ( X Y ) Var ( X ) Var ( Y ) 2 2 s s x y n m 2 s s x SE( X Y ) n m 2 y 11

12 Rescaled Test Statistic Test statistic: X Y Estimated standard error: s n 2 x x s n 2 y y Rescaled test statistic: t X Y s n 2 x x 0 s n 2 y y 12

13 For some test statistic T where m and s are not known, compute where Remember: Rescaling t T m m * is the hypothesized true value s T is the sample standard error of the statistic T s T * 13

14 One-sample and Two-sample Tests In a one-sample test of, choose m* Then T =, so the test statistic is In a two-sample test, you re often testing whether the means are equal T = t t X * * T m X m s. d.( T) s. e.( X ) X Y, and the test statistic is * T m ( X Y ) 0 ( X Y ) s. d.( T) s. e.( X Y ) s. e.( X Y ) 14

15 Equal Variances? We must estimate s x and s y If s x = s y then we can get a better estimate Remember: Sample variance for a single sample: s 2 n 1 1 Average squared deviation from the mean n ( x j j x 1 ) 2 Sample mean Deviations from sample mean 15

16 Different Means But Similar SD Remember, SD is calculated using differences from the mean Each group can have very different mean but standard deviations can be similar

17 More Bang for the Buck Pooled estimate of sample variance: Sample mean for process a Sample mean for process b s 2 p n 2 m 2 ( x ) ( ) 1 j x y j j 1 j y ( n 1) ( m 1) Average squared deviation from different means Used two degrees of freedom, n+m-2 left over Pooled estimate buys you more df Weighted average of 2 s x and s 2 y 17

18 Conducting the Test in JMP Need two variables: one continuous and one categorical (denoting group) Then: Analyze > Fit Y by X (continuous variable is the Y and categorical the X) > red triangle > Means/Anova/Pooled t See the t Test part of the output 18

19 Case: Taste Testing Teas Small taste test of teas (taste.jmp) 16 panelists in a focus group Each tasted two formulations of a prepackaged iced tea Rated them on a scale of 1 (excellent) to 7 (really bad) Company wants to know if there is a difference in ratings between the two formulations 19

20 An Initial Evaluation Two-sample t-test on taste.jmp: Is there a significant difference? 20

21 Taste Case: Any Difference? Unless SD s vastly different (factor of 2), the equal variance assumption no big deal 21

22 Independence Assumption Very Important Independence assumption for two sample t-test is violated Good news: there is an alternate test that can do even better Paired t-test assumes two observations taken for each unit in the sample Observations on the same unit likely to be more similar than obs ns on different units Here same person tasted each formulation 22

23 Paired t-test Looks at Differences x 1 -y 1 =d 1 x 2 -y 2 =d 2. x n -y n =d n Calculate differences for each observation Calculate sample mean and SD of differences Do a one sample t-test for differences: H 0 : mean difference is zero H a : mean difference is not 0 23

24 Paired t-test in JMP Use Analyze > Matched Pairs Two variables, paired by row: 24

25 Results: Paired t-test in JMP Mean Difference is same as two sample test SE is smaller why?? 25

26 Heuristic: Why Pairing Helps When x j and y j vary together then y j will be big when x j is big Since x j & y j tend to be close together, x j -y j is smaller than when X and Y independent Math: When X and Y are not independent then Var( X Y ) Var( X ) Var( Y ) 2 Cov( X, Y ) Cov or covariance measures linear dependence between two variables 26

27 It Helps in this Case Because People first have a like or dislike for tea Their ratings of the formulations are relative to this overall opinion of tea Taking the difference removes the person effect Tend to dislike tea Taste Taste 2 Tend to like tea 27

28 Independence vs. Dependence X X x j -y j is horizontal distance to the y=x line x j -y j is smaller (typically) in the right hand plot 28

29 Case: Sales Force Comparison Newly merged pharmaceutical company (PharmSal.jmp) Two sales forces ( BW & GL ), one from each of the merged companies 20 sales districts are the same Sales reps divided into these districts Sell essentially the same drugs Management wants to know if one sales force outperforms the other 29

30 Sales by Division Sales BW GL Division Quantiles Level Minimum 10% 25% Median 75% 90% Maximum BW GL

31 Two-Sample t-test Results Sales BW GL Under the independence assumption, we conclude that there is no difference in the means But are they independent? Division 31

32 The Sales Forces Are Dependent Dependence occurs by sales district: 32

33 Paired t-test Comparison Which sales force is doing better? 33

34 More Complicated Tests There are even more complicated tests you can do E.G., test for equal variance You re never going to remember all the steps for each test anyway Let the computer do it for you 34

35 Terminology One-sided vs. two-sided Comes from the statement of the alternative hypothesis Are you calculating the p-value using one tail or two? One-sample vs. two-sample Comes from the type of data and the question you are answering Are you testing a mean or a difference between means? 35

36 Which Test? How many populations are sampled? One: one-sample test Two: read on Are observations in first sample independent of observations in second sample? Yes: two-sample t-test No: paired t-test Big Clue: Paired t-test needs two observations from each unit Unequal sample sizes 2 sample test Equal sample sizes you have to decide 36

37 Hypothesis Tests in the Computer Age Know the null and alternative hypotheses Have some idea of what test statistics you would look at Let the computer figure out how to rescale them Let the computer figure out the p-value p-values are always interpreted the same way 37

38 What we have learned so far Descriptive Statistics Probability Inference for a population mean Confidence intervals Hypothesis testing One-sample test of the mean Two-sample tests Paired tests 38