Notes on Hypothesis Testing

Transcription

1 Probability & Statistics Grishpa Notes o Hypothesis Testig A radom sample X = X 1,..., X is observed, with joit pmf/pdf f θ x 1,..., x. The values x = x 1,..., x of X lie i some sample space X. The parameter parameter tuple θ is ukow, varies i some parameter space Θ. Partitio Θ ito two disjoit subsets H ad H 1. H is the ull hypothesis ad H 1 is the alterative hypothesis. A hypothesis test is a partitio of X ito two disjoit subsets, A ad R. A is the acceptace regio ad R is the rejectio regio. If X = x A is observed, we accept H, ad if X = x R is observed, we reject H. If θ H, the P θ X R = P θ rejectio is the type I error probability. If θ H 1, the P θ X A = P θ acceptace is the type II error probability. We wat to choose A ad R so that both error probabilities are small. Example A coi of ukow bias is tossed 1 times. X umber of heads Θ = { < θ < 1} X = {, 1,,..., 1} θ probability of heads H : θ =.5 A = {4, 5, 6} f θ x = 1 x θ x 1 θ 1 x H 1 : θ.5 R = {, 1,, 3, 7, 8, 9, 1}. Type I error probability, θ =.5 Type II error probability, θ.5 P.5 R = 1 1 x 1 P θ A = 1 θ x 1 θ 1 x x x R x A = θ 4 1 θ = θ 5 1 θ = = 4 θ4 1 θ 4 5 4θ1 θ < θ 6 1 θ 4 6 The sigificace level of a hypothesis test is the highest type I error probability, A upper boud o α is usually imposed. α = sup θ H P θ X R. To defie a decisio rule, we choose the form of the test. For istace, accept H wheever c 1 < T X < c or reject H wheever T X > c. Here T is a fuctio of the observed data, a test statistic. The costats c 1, c, c are critical values.

2 Samplig from ormal data: two-tailed z-test for mea Suppose that X i are idepedetly sampled from N θ, σ, ad we wat to test H : θ = θ vs H 1 : θ θ. Let us choose the test statistic to be T = X ad the form of the test to be accept H wheever c 1 < T < c. This is logical sice ay observatio X θ is i favor of H. The ull hypothesis i this case is simple as it cosists of oly oe value of θ. The alterative hypothesis is composite. We have α = P θ T c 1 or T c. The distributio of T that is used to determie α is ull distributio of T. The tail probabilities P θ T c 1 ad P θ T c should add up to α. Oe way to achieve this is to make them of equal weight, P θ T c 1 = P θ T c = α/, so that c 1 ad c are the lower ad upper α/ quatiles of ull distributio of T. Sice ull distributio of T is N θ, σ /, we have where z α c 1 = θ z α σ/ ad c = θ + z α σ/, is the upper α/ quatile for the stadard ormal distributio. For istace, if α =.5, the our test is defied by the decisio rule accept H wheever θ 1.96 σ/ < X < θ σ/. The same test ca be writte i terms of the statistic which has ull distributio N, 1 : Z = X θ σ/, accept H wheever Z < Note that the calculatio of critical values c 1, c ivolves oly type I error probability. Whe θ is a sigle variable, the power fuctio of a test with rejectio regio R is Power θ = P θ X R = P θ rejectio.

3 The plot of the power fuctio is a power curve. I our case, c1 θ Power θ = Φ σ/ + Φ c θ σ/ = Φ z α + θ θ σ/ + Φ z α + θ θ σ/ where Φ is the stadard ormal cdf. Observe that Power θ is symmetric about θ = θ, Power θ = α Power θ 1 as θ ±. is the miimum of Power θ, Samplig from ormal data: oe-tailed Suppose as above that X i z-test for mea are idepedetly sampled from N θ, σ, ad we wat to test H : θ θ vs H 1 : θ > θ. Here both hypotheses are composite. Choose the test statistic to be T = X ad the form of the test to be reject H wheever T > c. Sice the desity fuctio of T just slides to the right as θ icreases, we have α = max P θ T > c = P θ T > c = P Z > c θ θ θ σ/ = Φ c θ σ/. So c = θ + z α σ/, where z α is the upper α quatile for the stadard ormal distributio. The calculatio of the critical value c ivolves oly type I error probability. The power fuctio is Power θ = P θ R = Φ c θ σ/ = Φ z α + θ θ σ/.

4 Observe that Power θ is mootoically icreasig, Power θ = α, Power θ as θ, Power θ 1 as θ. If α =.5, the our test is defied by the decisio rule I terms of the statistic reject H wheever X > θ σ/. Z = X θ σ/, which has ull distributio N, 1, the same test ca be writte as reject H wheever Z > Samplig from ormal data: t-test for mea Suppose ow that X i are idepedetly sampled from N µ, σ, where µ ad σ are both ukow, θ = µ, σ. Cosider the hypotheses Istead of usig the statistic H : µ = µ vs H 1 : µ µ. Z = X µ σ/, which applies whe σ is kow, we approximate σ by the sample stadard deviatio S = 1 X i X 1 ad use the Studet t statistic T = X µ S/, which has ull distributio t 1, Studet s t distributio with 1 degrees of freedom. So the decisio rule ca be, for istace, i=1 accept H wheever T < t 1, α/,

5 where t 1, α/ is the upper α/ quatile for t 1. Thus a t-test is a refiemet of the idea of testig µ = µ i the case where σ is ukow. We have t 1, α > z α ot by much for large, i compesatio for the icreased variability of the test statistic. If µ µ, the T = X µ S/ N δ, 1, χ 1 1 where δ = µ µ σ/ is a o-cetrality parameter. The distributio of T whe the ull hypothesis is ot true is called a o-cetral t distributio. We may cosider the rejectio probability as a fuctio of sole parameter, Power δ. G. Lorde, Notes o Hypothesis Testig. J. Rice, Statistics ad Data Aalysis. P µ,σ rejectio Refereces