MIDTERM EXAM - MATH 563, SPRING 2016

Transcription

1 MIDTERM EXAM - MATH 563, SPRING 2016 NAME: SOLUTION Exam rules: There are 5 problems o this exam. You must show all work to receive credit, state ay theorems ad defiitios clearly. The istructor will NOT aswer ay questios durig the exam. Problem Poits / / / / / 15 TOTAL GRADE: (out of 90) 1

2 2 Problem 1. [This problem is about statistics ad data reductio techiques.] a) What does data reductio mea? I other words, how ca a statistic T (X), where X is a radom sample i the sample space X, be used for data reductio? A statistic T (X) is used for data reductio by partitioig the sample space X ito equivalece classes {X : T (X) = t}. Ay differet samples i the same equivalece class take the same value of T (X), ad are viewed as cotaiig the same amout of iformatio with regards to T. b) Defie a sufficiet ad acillary statistic. Let X be a radom sample from a populatio, whose distributio depeds o a parameter θ, A statistic T (X) is called sufficiet if the coditioal distributio of X give T (X) does ot deped o θ; A statistic S(X) is called acillary if its ow distributio does ot deped o θ. (cotiued o ext page)

3 Problem 1, cotiued. Solve oly ONE of the followig two problems, c) OR d): c) Let N be a radom variable takig values 1, 2,... with kow probabilities p 1, p 2,..., where i p i = 1. Havig observed N =, perform Beroulli trials with success probability θ, obtaiig X successes. Prove that the pair (X, N) is miimal sufficiet ad N is acillary for θ. [Hit: use a criterio, ot the defiitio of miimal sufficiet.] d) Prove Basu s Theorem i the DISCRETE case: If T (X) is a complete ad miimal sufficiet statistic, the T (X) is idepedet of every acillary statistic. [Hit: Let S(X) be ay acillary statistic, ad show that P (S(X) = s T (X) = t) P (S(X) = s) = 0.] SOLUTION of c): This is Exercise 6.12(a) from textbook. See Solutio to HW#3 for its aswer. 3 SOLUTION of d): Let S(X) be ay acillary statistic. As is stated i the hit, to prove that T (X) ad S(X) are idepedet, it suffices to show that (1) P (S(X) = s T (X) = t) P (S(X) = s) = 0 Sice S(X) is acillary, P (S(X) = s) does ot deped o θ. Also, sice T (X) is sufficiet, P (S(X) = s T (X) = t) does ot deped o θ. Note that (2) P (S(X) = s) = t P (S(X) = s T (X) = t)p θ (T (X) = t) Also, sice t P θ(t (X) = t) = 1, we have (3) P (S(X) = s) = t P (S(X) = s)p θ (T (X) = t) If we substract (3) from (2), we obtai (4) [P (S(X) = s T (X) = t) P (S(X) = s)] P θ (T (X) = t) = 0 t Sice T (X) is a complete statistic, we ca obtai (1) from (4). This completes the proof.

4 4 Problem 2. [This problem is about computig poit estimators.] 1) Let Y 1,..., Y deote a radom sample from the probability desity fuctio { (θ + 1)y θ, for 0 < y < 1 f(y θ) = 0 otherwise. Fid the method of momets estimator for θ. We have E(Y θ) = yf(y θ)dy = 1 (θ + 0 1)yθ+1 dy = θ+1. θ+2 Solvig θ from E(Y θ) = Ȳ, we obtai ˆθ MoM = 1 2Ȳ. Ȳ 1 2) Let X 1,..., X be a radom sample from the pdf f θ (x) = θx θ 1, for x (0, 1) (a beta distributio). Compute the MLE of θ. The likelihood is L(θ X) = f(x i θ) = θ (x 1 x ) θ 1.. Thus the log likelihood is Thus l(θ X) θ l(θ X) = log L(θ X) = log θ + (θ 1) log x i = 0 θ + log x i = 0 ˆθ MLE = log x i It is also easy to verify 2 l(θ X) θ 2 < 0, thus l(θ X) attais its maximum at ˆθ MLE. (cotiued o ext page)

5 5 Problem 2, cotiued. 3) Let X,..., X be a radom sample from P oisso(λ). Sice Gamma is a cojugate family for Poisso, let λ have a Gamma distributio. As writte o the board durig the exam, the X i are meat to just be Y i here. a) Write dow the formula you would use for computig the posterior distributio of λ, ad explai the meaig of each term i your formula. [Please do NOT write dow the formulas for the desities!] b) Assume that the calculatio i part a) would provide that the posterior distributio of λ is β β+1 Gamma(y + α, What would you use as the Bayes estimator for λ? (a) The formula used here is π(λ y) = f(y λ)π(λ) m(y). β ), with E[λ y] = (y + α) ad V ar(λ y) = (y + a) β 2 β+1 (β+1) 2. π(λ y) is the posterior desity of λ, which is the desity estimated after the data is observed. π(λ) is the prior desity of λ, which is the desity set before the data is collected. f(y λ) is the samplig desity of data give λ. m(y) = f(y λ)π(λ)dλ is the margial distributio of data. (b) We ca use E(λ y) = (y+α)β β+1 as the atural Bayesia estimator.

6 6 Problem 3. [This problem is about evaluatig poit estimators.] Let X 1,..., X be a radom sample from Beroulli(θ). 1 The, T = X i is a sufficiet statistic. The goal of this problem is to estimate η = θ(1 θ). a) Show that the aive estimator η = X 1 (1 X 2 ) is ubiased. Sice X 1 ad X 2 are idepedet, so are X 1 ad 1 X 2. We the have. Thus η is ubiased. E( η) = E[X 1 (1 X 2 )] = (EX 1 )(1 EX 2 ) = θ(1 θ) = η b) Fid a better estimator ˆη = E[ η T = t] usig the Rao-Blackwell Theorem. (Hit: The radom variable X 1 (1 X 2 ) is either 0 or 1; it s 1 if ad oly if X 1 = 1 ad X 2 = 0.) Accordig to the hit, we have ˆη =E( η T = t) =P ( η = 0 T = t) 0 + P ( η = 1 T = t) 1 =P ( η = 1 T = t) =P (X 1 = 1, X 2 = 0 X i = t) = P (X 1 = 1, X 2 = 0, X i = t) P ( X i = t) = P (X 1 = 1)P (X 2 = 0)P ( i=3 X i = t 1) P ( X i = t) = θ(1 θ)( ) 2 t 1 θ t 1 (1 θ) ( 2) (t 1) ) θt (1 θ) t = ( 2 t 1 ( t) ) t( t) = ( 1) ( t c) What is a miimum variace ubiased estimator (MVUE) of η? Could it be that ˆη is a MVUE? Accordig to Rao-Blackwell Theorem, if η is a ubiased estimator of η (which was prove i part a), ad if T (X) is a sufficiet statistic (which is give i the problem), the E( η T ) is a MVUE. Therefore, yes, ˆη computed i (b) is a MVUE. 1 Recall that θ = E[X i ] i this case.

7 7 Problem 4. [This problem is about covergece.] Let X 1,..., X deote a radom sample from a distributio with mea µ ad variace σ 2. a) State the Weak Law of Large Numbers. (Remider: this is a statemet about the sample mea X.) WLLN states that if σ 2 <, the X P µ as. This meas lim P ( X µ ɛ) = 1. Also, see P232, Theorem i the textbook. b) Show that the sample variace S 2 is a cosistet estimator of σ 2. We eed to assume Var(X 2 i ) < for i = 1,,. The by WLLN, as, we have (5) X P µ (6) Thus 1 X 2 i S 2 = 1 1 P E(X 2 i ) = µ 2 + σ 2. (X i X ) 2 = 1 1 ( Xi 2 X ) 2 = 1 ( 1 From (5) ad (6), ad the fact that lim, This completes the proof. S 2 1 Xi 2 ) 1 X 2 = 1, we ca coclude from Slutsky s Theorem that as P (µ 2 + σ 2 ) µ 2 = σ 2

8 8 Problem 5. [This problem is about evaluatig poit estimators asymptotically.] Let X 1,..., X be a radom sample from Beroulli(p). 2 Recall that the MLE of p is ˆp = X ad that it is a ubiased estimator. Show that the MLE attais the Cramer-Rao lower boud. Note that this example is exactly from the lectures (whe the Fisher iformatio was itroduced ad whe the example of the Cramer-Rao lower boud was computed). Here is the complete aswer: From Corollary (P337), we eed to prove Var( X) = ( d dp E X) 2 E[( p log f(x p))2 ]. Sice Beroulli distributio belogs to the expoetial family, from Lemma (P338), it is equivalet to show ( d dp (7) Var( X) = E X) 2 E[ 2 log f(x p)] p 2 Sice X i Beroulli(p), we have EX i = p, Var(X i ) = p(1 p), for i = 1,,. Thus E X = EX i = p, Var( X) = 1 Var(X i) = p(1 p). We the have d (8) dp E X = 1 (9) Var( X) = Also, p(1 p) log f(x p) = log[p X (1 p 1 X )] = X log p + (1 X) log(1 p), p log f(x p) = X p 1 X 1 p 2 p log f(x p) = X 2 p 1 X 2 (1 p) 2 Bearig i mid that EX = p, we have (10) E[ 2 p 2 log f(x p)] = 1 p 1 1 p = 1 p(1 p) Takig (8) ad (10) ito (7), we fid the right side of it to be 1 is the left side of (7). This completes our proof of (7). p(1 p) = p(1 p), which, accordig to (9), 2 Recall that E[X i ] = p ad V ar(x i ) = p(1 p).