Practice problems for Homework 11 - Point Estimation

Transcription

1 Practice problems for Homework 11 - Point Estimation 1. (10 marks) Suppose we want to select a random sample of size 5 from the current CS 3341 students. Which of the following strategies is the best: a) Pick 5 students from the first row. b) Pick 5 of your friends from the class. c) Post a note on elearning and get the first 5 students who respond. d) Wait near the door and pick the first 5 students who enter the class. e) Assign a number to each student in the class and use a random number generator to pick 5 students.. (10 marks) Consider the following results of 10 tosses of a coin: H, T, T, T, T, H, T, H, T, T a) Estimate the probability of head (H) for this coin. b) Estimate the standard error of your estimate. 3. (10 marks) Suppose the following data shows the number of the problems from the Practice Problems Set attempted in the past week by 10 randomly selected students: a) Find the sample mean. b) Find the sample variance., 4, 0, 7, 1,, 0, 3,, 1. c) Estimate the mean number of practice problems attempted by a student in the past week. d) Estimate the standard error of the estimated mean. 4. (10 marks) Let X 1,..., X n denote a random sample from a Uniform (0, θ) distribution, with θ > 0 as the unknown parameter. Let X denote the sample mean. a) Is X unbiased for θ? Explain your answer. b) Find an unbiased estimator for θ. c) Find the variance of the estimator in the previous part. d) Suggest an alternative estimator for θ. 1

2 5. (10 marks) A sample of 3 observations, (X 1 = 0.4, X = 0.7, X 3 = 0.9) is collected from a continuous distribution with density Estimate θ a. by the method of moments; b. by the method of maximum likelihood. 6. (10 marks) (P. 648, #3, first part) f(x) = θx θ 1 for 0 < x < 1. The memory residence times of 13,171 jobs were measured, and the sample mean was found to be 0.05 s and the sample variance, s. Assuming that the memory residence time is gamma-distributed, estimate its parameters r and λ using the method of moments. 7. (10 marks) Derive method of moments and maximum-likelihood estimators for: a) parameter p based on a Bernoulli(p) sample of size n. b) parameter p based on a Binomial(N, p) sample of size n. Compute your estimators if the observed sample is (3, 6,, 0, 0, 3) and N = 10. c) parameter λ based on a Poisson(λ) sample of size n. d) parameters a and b based on a Uniform (a, b) sample of size n. e) parameter µ based on a Normal(µ, σ ) sample of size n with known variance σ and unknown mean µ. f) parameter σ based on a Normal(µ, σ ) sample of size n with known mean µ and unknown variance σ. g) parameters (µ, σ ) based on a Normal(µ, σ ) sample of size n with unknown mean µ and variance σ.

3 Solutions: Strategy (e) is the best as this is the only strategy that ensures that each possible sample of 5 students is equally likely to be selected. The other strategies lead to convenience samples. Let X denote the toss of a single coin. Further, let X = 1 if a head results, and X = 0 if a tail results. This X is a Bernoulli (p) random variable, where p denotes the probability of head. Let ˆp denote the estimator of p. a) The estimated value of p is ˆp = ( )/10 = 0.3. b) The estimated standard error of ˆp is ˆp(1 ˆp)/n = 0.3(0.7)/10 = a) X = n X i/n = ( )/10 =. b) S = n (X i X) /(n 1) = ( (.) + (4.) (.) + (1.) ) /(10 1) = 4.4 c) The estimate is X =.. d) Estimated standard error of X is S/ n = 4.4/10 = Let X denote a Uniform (0, θ) random variable. a) No. This is because expectation of E(X) is θ/, which implies that E(X) = E(X) = θ/ θ. b) The estimator X is unbiased since E(X) = θ, for all θ. c) var(x) = 4var(X) = 4var(X)/n = 4θ /(1n) = θ /(3n). d) Since θ denotes the maximum value in the population, a natural estimator is maximum value in the sample, i.e., maxx 1,..., X n }. 5. a. Method of moments. Compute 3

4 equate it to µ 1 = E(X) = 1 0 xf(x)dx = 1 0 θx θ dx = θxθ+1 θ + 1 x=1 x=0 = θ θ + 1, and solve for θ, m 1 = X = = 3 θ θ + 1 = 3 = ˆθ = b. Method of maximum likelihood. The joint density is Take logarithm, f(x 1,..., X n ) = 3 θx θ 1 i 3 3 ln f(x 1,..., X n ) = ln θ + (θ 1) ln x i } = 3 ln θ + (θ 1) ln x i Take derivative, equate to 0, and solve for θ, ln f θ = 3 3 θ + ln x i = 0, 3 ˆθ = 3/ ln x i = 3/(ln ln ln 0.9) = For the Gamma distribution, E(X) = r/λ and V ar(x) = r/λ. So, the first two moments are µ 1 = r/λ and µ = (r/λ ) + (r/λ). 4

5 Also, the first two sample moments are M 1 = X and M = 1 n X i. Then, the method of moments estimators can be obtained by solving the system of equations r/λ = M 1 with respect to r and λ. µ = (r/λ ) + (r/λ) = M r/λ = M 1 (r/λ ) + (r/λ) = M r/λ = M 1 r/λ + M 1 = M r = M 1 /(M M 1 ) λ = M 1 /(M M 1 ) Notice that M M 1 = 1 n (Xi X) = (n 1)s /n = (13170)( )/(13171) = Then we can compute the estimates, ˆr = X / = 0.37, ˆλ = X/ = It is easier to use the \bf central} second moments, s and V ar(x) instead of E(X ) and M. This way, we solve the system of equations E(X) = r/λ = X V ar(x) = r/λ = s. Solving this system in terms of r and λ, we immediately get the method of moment estimates ˆr = X /s = 0.37, ˆλ = X/s = # 1a (Bernoulli) This is a special case of #1(b) when N = 1. 5

6 Following #1(b), both methods result in ˆp = X. # 1b (Binomial) Method of moments. To estimate parameter p of Binomial(N, p) distribution, we recall that µ 1 = E(X) = Np. There is only one unknown parameter p, hence we write one equation. Equate µ 1 = E(X) = Np to X and solve: ˆp = X/N. Maximum likelihood. Start with the joint p.m.f. f(x 1,..., X n ) = n ( N X i ) p X i (1 p) N X i, then ln f(x 1,..., X n ) = ( N X i ) + X i ln p + (N X i ) ln(1 p), p ln f(x 1,..., X n ) = n 1 X n i 1 (N X i) p 1 p (find roots of the derivative in order to maximize the density). Solving this equation, = n X p nn n X 1 p = 0, Answer: ˆp = X/N. (1 p) X = p(n X) p = X/N. # 1(c) (Poisson) Method of moments. To estimate parameter λ of Poisson(λ) distribution, we recall that µ 1 = E(X) = λ. There is only one unknown parameter, hence we write one equation, µ 1 = λ = m 1 = X. 6

7 Solving it for λ, we obtain ˆλ = X, the method of moments estimator of λ. Maximum likelihood. The p.m.f. of Poisson distribution is P (x) = e λ λx x!, and its logarithm is ln P (x) = λ + x ln λ ln(x!). Thus, we need to maximize ln P (X 1,..., X n ) = ( λ + X i ln λ) + C = nλ + ln λ X i, where C = ln(x!) is a constant that does not contain the unknown parameter λ. Find the critical point(s) of this log-likelihood. Differentiating it and equating its derivative to 0, we get This equation has only one solution λ ln P (X 1,..., X n ) = n + 1 λ ˆλ = 1 n X i = X. X i = 0. Since this is the only critical point, and since the likelihood vanishes (converges to 0) as λ 0 or λ, we conclude that ˆλ is the maximizer. Therefore, it is the maximum likelihood estimator of λ. For the Poisson distribution, the method of moments and the method of maximum likelihood returned the same estimator, ˆλ = X. # 1d (Uniform) 7

8 Method of moments. Use two moments, µ 1 = (a + b)/ and, for example, µ = V ar(x) = (b a) /1. Find â and ˆb by solving the system a+b = X (b a) = S â = X S 1 1, ˆb = X + S 1 Maximum likelihood. The joint density f(x 1,..., X n ) = ( 1 b a) n if a X 1,..., X n b 0 otherwise is monotonically increasing in a and decreasing in b. It is maximized at the largest value of a and the smallest value of b where this density is not 0. These are â = min(x i ) and ˆb = max(x i ) # 1e (Normal with unknown mean) Method of moments. Equate µ to X and trivially obtain ˆµ = X. Maximum likelihood. The joint density is Then f(x 1,..., X n ) = n 1 σ π exp (X i µ) σ }. ln f(x 1,..., X n ) = n ln(σ π) (X i µ) σ = n ln(σ π) nµ X i µ + Xi σ is a parabola in terms of µ, and it is maximized at ˆµ = X i /n = X. # 1f (Normal with unknown variance) Method of moments. The first population moment is not a function of σ, thus we equate the second (central, for simplicity) moment σ to the second moment S and obtain ˆσ = S. 8

9 Since µ is known, we can also use the estimator instead of S. ˆσ = 1 n (X i µ) Maximum likelihood. Differentiate ln f(x 1,..., X n ) from #1e with respect to σ, σ ln f(x 1,..., X n ) = n σ + (Xi µ) = 0, σ 3 (find roots of the derivative in order to maximize the density) so that #1 g (Normal with both parameters unknown) Method of moments. (Xi µ) ˆσ =. n When both µ and σ are unknown, we equate the first and second (central, for simplicity) moments and trivially obtain ˆµ = X and ˆσ = S so that ˆσ = S. Maximum likelihood. First, maximizing ln f(x 1,..., X n ) from # 1e in µ, we again obtain ˆµ = X regardless of the value of σ. Then, substitute the obtained maximizer into ln f(x 1,..., X n ) for the unknown value of µ and maximize the resulting function in terms of σ. We get the same answer as in (d) with the unknown parameter µ replaced by X, (Xi ˆσ = X) = S. n 9