Lecture 10: Hypothesis testing and confidence intervals

Transcription

1 Eco 514: Probability ad Statistics Lecture 10: Hypothesis testig ad cofidece itervals Types of reasoig Deductive reasoig: Start with statemets that are assumed to be true ad use rules of logic to esure that all derived statemets are also true. This is doe i mathematical reasoig. Iductive reasoig: Start with statemets that are assumed to be true ad use the rules of logic to derive true implicatios that ca be verified by empirical observatio. If verificatio fails, this casts doubt o the iitial statemets ad their truth. Verificatio by empirical observatio ca be coclusive, as i the observatio of the relatio betwee temperature ad pressure of a gas i a closed chamber, or it ca be icoclusive. We will be dealig with the latter case. 1

2 Hypothesis testig If the statemet that has to be verified is a statemet about a populatio (distributio), the we must obtai a (radom) sample from this populatio. With a radom sample we caot be absolutely certai that the statemet is true or false, because of samplig variatio. Hece ay decisio o the truth of the statemet is a decisio uder ucertaity. The part of statistics that deals with the verificatio of statemets about a populatio is called hypothesis testig. 2

3 Example: Cosider the populatio of LA households i two years. The populatio icome distributio i these years is N(µ 1, σ 2 ) ad N(µ 2, σ 2 ) respectively. Cosider the statemet/cojecture about the populatio: The mea icome is the same i both years. This ca be expressed as µ 1 = µ 2 A cojecture/statemet about (a) populatio distributio(s) is called a statistical hypothesis. We could have the statistical hypothesis µ 1 = µ 2 = $1800 (per moth) ad if σ 2 were kow or if we also specified its value as part of the hypothesis, the the populatio distributio is completely specified by the hypothesis. Such a statistical hypothesis is called simple. Otherwise it is called composite. 3

4 Notatio for statistical hypothesis: H 0. Example or H 0 : µ 1 = µ 2 H 0 : µ 1 = µ 2 = 1800 This is also called the ull hypothesis. The complemet of the ull hypothesis, i.e. ot H 0, is called the alterative hypothesis. I above example: H 1 : µ 1 µ 2 or H 1 : µ 1 µ 2 or µ 1 = µ

5 Hypothesis testig as a decisio problem A test of H 0 is a decisio rule for the decisio whether to reject H 0. Rejectio of H 0 is iformative because Theory Implicatio H 0 H 0 is rejected Theory is false Coclusio: If H 0 is false, the the theory is false. Decisio rules The decisio rule tells us to pick H 0 or H 1. Nature of decisio rule: If the observed sample is x 1,..., x, a set C R is specified. We reject H 0 if (x 1,..., x ) C. C is called the critical or rejectio set of the test. 5

6 Example: Testig equality of populatio meas. Let the icome distributio i 2 years be N(µ 1, σ 2 ) ad N(µ 2, σ 2 ) withσ 2 kow. We test H 0 : µ 1 = µ 2 The radom samples i the two years are X 1,..., X aad Y 1,..., Y, respectively (ote equal size). Reasoable to base test o T (X 1,..., X, Y 1,..., Y ) = X Y 2σ 2 If H 0 : µ 1 = µ 2 is true, the T N(0, 1) Hece it is reasoable to take as rejectio set C = {(x 1,..., x, y 1,..., y ) T (x 1,..., x, y 1,..., y ) > c} Why ad why symmetric aroud 0? This leaves the questio how to choose c. 6

7 Decisio errors ad choice of rejectio regio X 1,..., X is a radom sample from a populatio with desity f(x; θ) with θ Θ a ukow populatio parameter. Statistical hypothesis H 0 : θ Θ 0 H 1 : θ Θ 1 = Θ \ Θ 0 ull hypothesis alterative hypothesis Test specifies critical/rejectio regio C R with Reject H 0 (x 1,..., x ) C Decisios ad state of the world State of the world Decisio H 0 true (H 1 false) H 1 true (H 0 false) Not reject H 0 OK Type II error Reject H 0 Type I error OK Questios: How do we evaluate tests? How do we fid a good test? 7

8 Evaluatio of tests I estimatio we cosidered the estimatio error ˆθ θ This is measured o a clear scale. I testig problems we have type I ad type II errors. These errors are hard to compare. Example: Test for presece of disease H 0 : diseased H 1 : healthy Type I error (false rejectio): false peace of mid. Type II error (false cofirmatio): false alarm. Which is worse? Type I ad II errors are depedet: if we decrease the probability of a type I error, we icrease the probability of a type II error. Make e.g. the cutoff poit c large i the previous example. 8

9 Power fuctio The quality of a test is measured by the power fuctio π(θ) = P (rejecth 0 ; θ) = Pr((X 1,..., X ) C; θ) This is a fuctio of the populatio parameter θ. Ideal power fuctio (see figure) This caot be achieved: Compare with Cramér-Rao lower boud i estimatio. 9

10 Example 1 : Assume populatio icome distributio i LA is N(θ, 1) i 1000$/moth. We have composite ull ad alterative H 0 : θ < 2 H 1 : θ 2 Reasoable test has rejectio set C { } C = (x 1,..., x ) 1 x i 2 Because X N i=1 ( θ, 1 ) Hece (see figure) π(θ) = Pr(X 2) = Pr X θ 1 2 θ 1 = 1 Φ( (2 θ)) Note If θ close to 2, the probability of (type I or II) error is close to.5. If is large the power fuctio approaches the ideal power fuctio. 10

11 Example 2: Simple ull hypothesis H 0 : θ < 2 Rejectio set { C = (x 1,..., x ) 1 Hece (see figure) H 1 : θ 2 x i 3or i=1 } x i 1 i=1 π(θ) = 1 Pr(1 < X < 3) = 1 Pr 1 θ 1 < X θ 1 < 3 θ 1 = = Φ( (1 θ)) + 1 Φ( (3 θ)) 11

12 Note It does ot make sese to use the rejectio regio { } C = (x 1,..., x ) 1 x i 2 i=1 If, the π(θ) approaches the ideal power fuctio. 12

13 Optimal test I our defiitio of a optimal test we fix the type I error ad miimize the type II error give the type I error. Defiitio: A test is uiformly most powerful amog all tests of size α of H 0 : θ Θ 0 agaist H 1 : θ Θ 1 if its power fuctio π(θ) satisfies (i) sup θ Θ0 π(θ) = α (ii) π(θ) π (θ) for all θ Θ 1 ad all tests with power fuctio π (θ) such that sup θ Θ0 π (θ) α. sup θ Θ0 is called the size of the test. Hece a UMP test has greater power o H 1 amog all tests with size less equal α. Neyma-Pearso lemma: Simple ull ad alterative. Let Θ = {θ 0, θ 1 } ad H 0 : θ = θ 0 H 1 : θ = θ 1 This amouts to choosig betwee two populatios with desities f(x; θ 0 ) ad f(x; θ 1 ). Cosider a radom sample of size 1 with observed value x. Reasoable rejectio set { C = {x f(x; θ 1 ) f(x; θ 0 )} = x f(x; θ } 1) f(x; θ 0 ) 1 13

14 If we have a radom sample of size we cosider i=1 λ(x 1,..., x ) = f(x i; θ 0 ) i=1 f(x i; θ 1 ) For obvious reasos λ is called the likelihood ratio ad a test based o λ is called a likelihood ratio (LR) test. We reject H 0 is λ is smaller tha a critical value. Because both H 0 ad H 1 are simple, we have P (Type I error) = π(θ 0 ) P (Type II error) = 1 π(θ 1 ) For this case the UMP test satsifies (i) π(θ 0 ) = α (ii) π(θ 1 ) π (θ 1 ) for all tests with π (θ 0 ) α. Hece the UMP test maximizes π (θ 1 ) s.t. π (θ 0 ) α. 14

15 Theorem (Neyma-Pearso lemma): The test with rejectio set C = {(x 1,..., x ) λ(x 1,..., x ) k} with k such that for a radom sample X 1,..., X from f(x; θ 0 ) Pr(λ(X 1,..., X ) k) = α is a UMP test of H 0 : θ = θ 0 agaist H 1 : θ = θ 1. Ay other UMP test with π (θ 0 ) α has size α, i.e. π (θ 0 ) = α ad rejectio set C A with Pr((X 1,..., X ) A) = 0 both if the radom sample is from f(x; θ 0 ) ad if it is from f(x; θ 1 ). Proof: Cosider other test with rejectio set C 1 power fuctio π (θ), so that π (θ 0 ) π(θ 0 ) = α The (we cosider the case of a distributio that is absolutely cotiuous w.r.t. the Lebesgue measure) π(θ 1 ) π (θ 1 ) = f(x i ; θ 1 )dx 1... dx f(x i ; θ 1 )dx 1... dx C i=1 Because C = (C \ C 1 ) (C 1 C) ad C 1 = (C 1 \ C) (C 1 C) we have (we deote the likelihoods i θ 0 ad θ 1 by L 0 ad L 1, ad x deotes the vector with compoets x 1,..., x ) π(θ 1 ) π (θ 1 ) = L 1 (x)dx L 1 (x)dx C\C 1 C 1 \C C 1 i=1 15

16 O C we have L 1 1 k L 0 ad o C c we have L 1 < 1 k L 0, ad after subsitutio we have π(θ 1 ) π (θ 1 ) 1 L 0 (x)dx 1 L 0 (x)dx = k C\C 1 k C 1 \C (1) = 1 { } L 0 (x)dx L 0 (x)dx = 1 k C C 1 k (π(θ 0) π (θ 0 )) 0 Hece π(θ 1 ) π (θ 1 ) ad we coclude that the LR test satisfies the coditios for a UMP test. Next, cosider aother UMP test with rejectio set C 1. Because we already kow that the LR test with power π(θ 1 ) is UMP, we have π (θ 1 ) = π(θ 1 ) ad from (1) we have π (θ 0 ) = π(θ 0 ) = α. Defie the fuctio h(x) = (I C (x) I C1 (x))(kl 1 (x) L 0 (x)) with I C the idicator fuctio of the set C. Check that by the defiitio of C this fuctio is oegative. Hece 0 h(x)dx = C (kl 1 (x) L 0 (x))dx (kl 1 (x) L 0 (x))dx = C 1 = kπ(θ 1 ) π(θ 0 ) kπ (θ 0 )+π (θ 0 ) = k(π(θ 1 ) π (θ 1 )) = 0 Hece h(x) = 0 for all x. 16

17 From the defiitio of h this implies that for all x with kl 1 (x) L 0 (x) 0, we have I C (x) = I C1 (x). The set B c with kl 1 (x) L 0 (x) = 0 has probability 0 (why?), so that C 1 B = C B ad if x C 1 the either x C (if x is also i B) or x is i a set of probability 0. Optimality of LR test of simple ull ad alterative for other criteria a. Miimax criterium. We cosider the losses that are associated with decisios State of the world Decisio H 0 true (H 1 false) H 1 true (H 0 false) H 0 selected 0 Type II error: loss l II H 1 selected Type I error: loss l I 0 The expected loss R(θ) of the test of H 0 : θ = θ 0 agaist H 1 : θ = θ 1 is R(θ) = l I π(θ 0 ) if θ = θ 0 = l II (1 π(θ 1 )) if θ = θ 1 Now max{r(θ 0 ), R(θ 1 )} is the maximal expected loss associated with the test. A miimax test miimizes the maximal expected loss, i.e. for ay other test with expected losses R (θ 0 ), R (θ 1 ) max{r(θ 0 ), R(θ 1 )} max{r (θ 0 ), R (θ 1 )} 17

18 If for the LR test we choose the critical value k such that R(θ 0 ) = l I π(θ 0 ) = l II (1 π(θ 1 )) = R(θ 1 ) Hece a better miimax test has max{r (θ 0 ), R (θ 1 )} < max{r(θ 0 ), R(θ 1 )} = R(θ 0 ) = R(θ 1 ) so that R (θ 0 ) < R(θ 0 ) ad R (θ 1 ) < R(θ 1 ). This implies that π (θ 0 ) < π(θ 0 ) ad π (theta 1 ) > π(θ 1 ), which is impossible because the LR test is UMP. b. Bayesia test As i the miimax criterium we combie the expected losses R(θ 0 ), R(θ 1 ) is a scalar criterium. We assig probabilities/weights to the possible values of θ. These weights reflect a priori beliefs about the value of this parameter. If the weights are g ad 1 g, the R = gr(θ 0 ) + (1 g)r(θ 1 ) This is the Bayesia expected loss. The best test miimizes R. If we choose the cutoff value i the LR test as k = gl I (1 g)l II the a similar proof as i the miimax case shows that the LR is the optimal Bayesia test. 18

19 Example of LR test ad extesios We have a N(θ, 1) populatio ad we test The likelihood is L(θ) = The likelihood ratio is λ(x) = e 1 2 i=1 (x i 3) H 0 : θ = 2 H 1 : θ = 3 ( ) 1 e 1 2 i=1 (x i µ) 2 2π i=1 (x i 2) 2 = e 1 2 ( 2 i=1 x i+5) Note that λ(x) k is equivalet to 1 x i 5 2 l k = c i=1 so that the rejectio set is { C = (x 1,..., x ) 1 with c such that for θ = 2 or 1 } x i c i=1 Pr(X c) = Pr X 2 (c 2) = α c = Φ 1 (1 α) The iverse stadard ormal cdf is tabulated i the book. 19

20 This is the UMP test. Note that the alterative H 1 : θ = 3 does ot play a role i the derivatio of the UMP test, i.e. the test is uchaged if we pick ay value greater tha 2 for the alterative. Hece the test is also the UMP test for H 0 : θ = 2 H 1 : θ > 2 i.e. with a composite alterative. Note that the test chages if e.g. H 1 : θ = 1. The power fuctio of the LR test is π(θ) = Pr(X c) = 1 Φ( (c θ)) This fuctio icreases i θ, so that sup θ 2π(θ) = α Hece the LR test with rejectio set C has size α if H 0 : θ 2. Hece the test is also UMP for H 0 : θ 2 H 1 : θ > 2 i.e. a composite ull ad alterative. These tricks that exted the ull ad alterative hypotheses from simple to composite caot always be applied. I geeral, a UMP test of size α for composite ull ad alterative ca oly be obtaied i special cases. If the UMP test exists, it is a LR test. 20

21 Geeralized LR test The LR test ca be used as the ML to mechaically derive tests (that may or may ot be UMP).. We have a R vector of parameters θ 1 θ =. θ R ad cosider a hypothesis o a sub-vector H 0 : θ 1 = θ 10,..., θ r = θ r0 H 1 : θ 1 θ 10,..., θ r θ r0 The geeralized LR statistic is defied by λ(x 1,..., X ) = sup θ Θ,θ 1 =θ 10,...,θ r =θ r0 L(θ) sup θ Θ L(θ) The deomiator is L(ˆθ) with ˆθ the MLE. The umerator is L( θ) with θ 10. θ = θ r0 θ r+1 θ R ad θ r+1,..., θ R the restricted MLE of θ r+1,..., θ R. Note that 0 λ(x 1,..., X ) 1, so that 2 l λ(x 1,..., X ) 0. If λ(x 1,..., X ) is small, i.e. 2 l λ(x 1,..., X ) is large, we reject H 0. 21

22 Theorem: If H 0 is true, the 2 l λ(x 1,..., X ) d χ 2 (r) The rejectio set of the geeralized LR is C = {(x 1,..., x ) 2 l λ(x 1,..., x ) > c} with c such that Pr(X C) = Pr( 2 l λ(x) > c) = 1 F χ2 (r)(c) = α ad we fid c from the table of the chi-square distributio. Example: Test i N(µ, σ 2 ) populatio. We test H 0 : µ = µ 0 H 1 : µ µ 0 The log likelihood fuctio is l L(µ, σ 2 ) = 2 l 2π 2 l σ2 1 2σ 2 The (urestricted) MLE are ˆµ = 1 ˆσ 2 = 1 i=1 X i (X i X ) 2 i=1 The restricted MLE of σ 2 is σ 2 = 1 (X i µ 0 ) 2 i=1 (x i µ) 2 i=1 22

23 The geeralized LR statistic is ( i=1 λ(x) = (X i X ) 2 ) 2 i=1 (X i µ 0 ) 2 which ca be rewritte as ( i=1 λ(x) = (X i X ) 2 ) 2 i=1 (X i X ) 2 + (X µ 0 ) 2 Hece λ(x) k iff (X µ 0 ) 2 i=1 (X i X ) 2 c or (X µ 0 ) ˆσ c If we replace the MLE ˆσ by the ubiased S we have (X µ 0 ) S t( 1) so that a fiite sample size α test ca be foud from the table with the cdf of the t distributio. If, t( 1) d N(0, 1) ad t( 1) 2 d χ 2 (1) 23

24 Wald test Remember for the MLE (ˆθ θ 0 ) d N(0, σ 2 (θ)) with We test σ 2 (θ) = 1 ( ( ) ) 2 E l f θ (X 1; θ) H 0 : θ = θ 0 H 1 : θ θ 0 I aalogy to the last example we cosider the statistic if H 0 is true (ˆθ θ 0 ) d T = N(0, 1) σ(ˆθ ) which follows from Slutsky s theorem We reject if (ˆθ θ 0 ) σ(ˆθ ) > c where c is foud from the stadard ormal cdf table. We expect this to be a good test because the MLE has i large samples the smallest possible variace, equal to the Cramér-Rao lower boud. This test is a example of a Wald test, i.e. a test based directly o the asymptotic distributio of a estimator. 24

25 Cofidece itervals Cofidece itervals express the ucertaity associate with a parameter estimate. Istead of testig H 0 : θ = θ 0 we ca obtai a cofidece iterval for θ ad check whether θ 0 is i that iterval. Let X 1,..., X be a radom sample from N(θ, 1). X is a estimator of θ ad X N(θ, 1 ) If you plot this desity, you see that the observed sample mea x is has a high probability of beig ear θ, but i geeral it will ot be equal to θ. We use this by first observig that (X θ) N(0, 1). Note that this distributio does ot deped o ay ukow parameters. As a cosequece we ca for ay choice of a, b compute Pr(a < (X θ) < b) = Φ(b) Φ(a) ad this is the same as ( Pr X + a < θ < X + b ) = Φ(b) Φ(a) Hece i repeated samples (this is a thought experimet because i practice we have oly oe sample) we that [ θ X + a, X + b ] with probability Φ(b) Φ(a) = α. 25

26 If we choose e.g. α =.95, the θ will be with that probability i the specified (radom) iterval. To choose a, b ote that the legth of the iterval is proportioal to b a. Because we prefer a short iterval we choose a, b such that b a is miimal subject to Φ(b) Φ(a) = α. The first-order coditio implies φ(b) = φ(a) so that a = b. We fid b from (make a graph, if ecessary) Φ(b) = 1+α 2 usig the table of the stadard orma cdf. Hece the radom iterval is [ X b, X + b ] This is a cofidece iterval for θ with cofidece coefficiet or (coverage probability) 100α %. Defiitio: Let X 1,..., X be a radom sample from a populatio with pdf f(x; θ). Let L = L(X 1,..., X ) ad U = U(X 1,..., X ) are two statistics with Pr(L U) = 1. If Pr(L < θ < U) = α for all θ Θ, the [L, U] is a 100α % cofidece iterval for θ ad 100α is the cofidece coefficiet or α the coverage probability of the iterval. Fidig cofidece itervals Above we use the fact that the distributio of (X θ) does ot deped o θ. I geeral, cosider Q = q(x 1,..., X ; θ). If the distributio of Q does ot deped o θ we call it a pivotal quatity. It is ot a statistic, because it depeds o the ukow θ. 26

27 Pivotal quatities are helpful i fidig cofidece itervals. If Q is pivotal we ca fid q 1, q 2 that do ot deped o θ such that Pr(q 1 < Q < q 2 ) = α If the evet q 1 < q(x 1,..., X ; θ) < q 2 ca be rewritte as L(X 1,..., X ) < θ < U(X 1,..., X ) the [L, U] is a 100α % cofidece iterval for θ. Note that iversio ot always gives a coected cofidece iterval (see figure) 27

28 Cofidece itervals for the parameters of the ormal distributio X 1,..., X is a radom sample from N(µ, σ 2 ). For cofidece iterval for µ we use X µ σ N(0, 1) ( 1)S2 σ 2 χ 2 ( 1) X ad S 2 idepedet Hece Q = is pivotal. (X µ) σ ( 1)S 2 σ 2 1 = (X µ) S t( 1) Because the t-distributio is symmetric aroud 0, the cofidece iterval is [ X t S, X + t S ] ad t satisfies F t( 1) (t) = α+1 2 ad ca be foud usig the table of the t-distributio. For cofidece iterval for σ 2 we use Q = ( 1)S2 σ 2 χ 2 ( 1) χ 2 ( 1) which is a pivotal quatity. The cofidece iterval is [ ( 1)S 2, ( ] 1)S2 q 2 q 1 with q 1 < q 2. 28

29 The shortest iterval with coverage probability α is obtaied if we solve mi 1 q 1 1 q 2 subject to F χ2 ( 1)(q 2 ) F χ2 ( 1)(q 1 ) = α. This has to solved umerically. Cofidece iterval for differece of meas Cosider two radom samples from the icome distributio i LA i two years: X 1,..., X m ad Y 1,..., Y. The samples are draw idepedetly. We wat to fid a cofidece iterval for µ 2 µ 1. We have Y X m N(µ 2 µ 1, σ2 m + σ2 ) (m 1)S2 xm σ + ( 1)S2 y 2 σ χ 2 (m + 2) 2 These radom variables are idepedet. Hece Q = Y X m (µ 2 µ 1 ) σ 2 m + σ2 (m 1)Sxm 2 σ 2 + ( 1)S2 y σ 2 m+ 2 = Y X m (µ 2 µ 1 ) m (m 1)S 2 xm m+ 2 + ( 1)S2 y m+ 2 This is a pivotal quatity ad we fid the cofidece iterval as before. We also could have obtaied the icome from a radom sample of households i two years, i.e. a radom sample (X 1, Y 1 ),..., (X, Y ) from the bivariate ormal distributio with parameters µ 1, µ 2, σ 2 1, σ 2 2, ρ. This is called a pael survey. 29

30 Defie D i = Y i X i N(µ 2 µ 1, σ1 2 σ2 2 2ρσ 1 σ 2 so that (D (µ 2 µ 1 )) Q = t( 1) S D Q is pivotal ad we obtai a cofidece iterval as before. Compare the cofidece itervals for two idepedet samples (take m = ) ad the pael survey. Which is shorter? Large sample cofidece itervals If ˆθ is the MLE, the Q = (ˆθ θ) σ(ˆθ ) d N(0, 1) I large samples Q is pivotal ad we obtai a large sample cofidece iterval as [ ] ˆθ q σ(ˆθ ), ˆθ + q σ(ˆθ ) with Φ(q) = 1+α 2. 30