Bivariate Unit Normal. Lecture 19: Joint Distributions. Bivariate Unit Normal - Normalizing Constant. Bivariate Unit Normal, cont.

Transcription

1 Lecture 19: Joint Distributions Statistics 14 If X and Y have independent unit normal distributions then their joint distribution f (x, y) is given by f (x, y) φ(x)φ(y) c e 1 (x +y ) Colin Rundel April, 1 Statistics 14 (Colin Rundel) Lecture 19 April, 1 1 / 19, cont. - Normalizing Constant We can rewrite the joint distribution in terms of the distance r from the origin r x + y f (x, y) c e 1 (x +y ) c e 1 r Rewriting the joint distribution in this way also makes it easier to evaluate the constant of integration, c. (From our previous discussions of the normal distribution we know that c 1, but we did not see how to derive this explicitly) This tells us something useful about this special case of the bivariate normal distributions: it is rotationally symmetric about the origin, this particular fact is incredibly powerful and helps us solve a variety of problems. Y r r + eps P(R (r, r + ɛ)) π[(r + ɛ) r ]c e 1 r π[r + rɛ + ɛ r ]c e 1 r rɛc e 1 r f R (r) rc e 1 r, for r (, ) Statistics 14 (Colin Rundel) Lecture 19 April, 1 / 19 X Statistics 14 (Colin Rundel) Lecture 19 April, 1 3 / 19

2 - Normalizing Constant Rayleigh Distribution We can now solve for c 1 f R (r) dr rc e 1 r dr, Let u r πc e 1 u du c e 1 u c c 1 The distance from the origin of a point (x, y) derived from X, Y N (, 1) is called the Rayleigh distribution. f R (r) re 1 r F r (r) r re 1 r dr e 1 r r 1 e 1 r We will see the expected value in a little bit. Statistics 14 (Colin Rundel) Lecture 19 April, 1 4 / 19 Statistics 14 (Colin Rundel) Lecture 19 April, 1 5 / 19 Rayleigh Distribution - Example - Variance of the Normal If the position of a target shooter s shots are described by normal distributions in the x and y direction that are centered on the bull s eye with a standard deviation of 1 in: What is the probability a shot is more within 1 inch of the bull s eye? What is the probability a shot is more than inches away from the bull s eye? What is the probability a shot is between 1 and inches from the bull s eye? Assume we did not know that Var(X ) σ 1, what would need to evaluate to find the variance of X N (, 1)? Var(X ) E(X ) E(X ) E(X ) 1 x e 1 x dx We can use the Rayleigh distribution to make our life easier E(R ) E(X + Y ) E(X ) + E(Y ) E(X ) E(X ) E(R )/ Statistics 14 (Colin Rundel) Lecture 19 April, 1 6 / 19 Statistics 14 (Colin Rundel) Lecture 19 April, 1 7 / 19

3 - Variance of the Normal E(R ) is still not very easy to evaluate, but we can consider a change of variables where we let S R. / ds f S (s) f R (r) dr re 1 r /r 1 1 e s, for s (, ) Therefore, S has a exponential distribution with λ 1/. E(R ) E(S) 1/λ σ E(X ) E(R )/ 1 Rayleigh Distribution - Expectation We can also use the preceding result to help find the expected value of R. E(R) rf R (r)dr r e 1 r dr 1 x e 1 x dx x φ(x)dx E(X ) π 1 x e 1 x dx Statistics 14 (Colin Rundel) Lecture 19 April, 1 8 / 19 Statistics 14 (Colin Rundel) Lecture 19 April, 1 9 / 19 Alternative Approach Previously we have shown that the sum (or average) of iid random variables will converge to the normal distribution as the number of random variables goes to infinity thanks to the central limit theorem. Intuitively it makes sense then that the sum of a small number of normals should also be normal but we have not explicitly shown this to be true. Also, what about the case where the normals are not identically distributed? Based on arguments using projections and the s rotational symmetry the book derives the following general result (and special cases): Let X and Y be independent normals with distributions N (µ, σ ) and N (λ, τ ), then X + Y has a N (µ + λ, σ + τ ) distribution. a) If X, Y N (, 1), then αx + βy N (, 1) for all α and β with α + β 1 b) If X, Y N (, 1), then X + Y N (, ) It is also possible to prove these results using moment generating functions. Recall: Moment generating functions unique identify distributions. If X and Y are independent with mgfs M X (t) and M Y (t) then M X +Y (t) M X (t)m Y (t) If Z N (µ, σ ) then M Z (t) exp(µt + 1 σ t ) Statistics 14 (Colin Rundel) Lecture 19 April, 1 1 / 19 Statistics 14 (Colin Rundel) Lecture 19 April, 1 11 / 19

4 Proof of b) Let X, Y N (, 1) then Proof of a) Let X, Y N (, 1) and define α and β such that α + β 1 then M X (t) M Y (t) exp( 1 t ) M X +Y (t) exp( 1 t ) exp( 1 t ) exp( 1 t ) This is the moment generating function of a normal distribution with µ and σ, therefore X + Y N (, ). αx N (, α ) βy N (, β ) M αx (t) exp( 1 α t ) M βy (t) exp( 1 β t ) M αx +βy (t) exp( 1 α t ) exp( 1 β t ) exp( 1 (α + β )t ) exp( 1 t ) Therefore αx + βy N (, 1). Statistics 14 (Colin Rundel) Lecture 19 April, 1 1 / 19 Statistics 14 (Colin Rundel) Lecture 19 April, 1 13 / 19 Proof of General Result Let X N (µ, σ ) and Y N (λ, τ) then M X (t) exp(µt + 1 σ t ) M Y (t) exp(λt + 1 τ t ) M X +Y (t) exp(µt + 1 σ t ) exp(λt + 1 τ t ) exp((µ + λ)t + 1 (σ + τ )t Further Generalization and Example If X, Y, Z are all independent normal random variables then we have shown that X X + Y must also be a normal random variable and it follows that Z X + Z must also be a normal random variable, which is equivalent to X + Y + Z which is therefore a normal random variable. Example: If X, Y, Z N (, 1) then what is P(X + Y < Z + )? Therefore X + Y N (µ + λ, σ + τ ). Statistics 14 (Colin Rundel) Lecture 19 April, 1 14 / 19 Statistics 14 (Colin Rundel) Lecture 19 April, 1 15 / 19

5 Multivariate Unit Normal Distribution of R n Let Z 1,..., Z n N (, 1) then the joint distribution of (z 1,..., z n ) is f (z 1,..., z n ) f (z 1 ) f (z n ) ( ) n 1 exp[ 1 (z zn )] Once again we can model the distance of a point from the origin in this n-dimensional space using R Z Z n P(R n (r, r + ɛ)) c n ɛr n 1 ( 1 ) n e 1 r where c n is the (n 1)-dimensional volume of the surface of a sphere of radius 1 in n-dimensions. Eg. c and c 3 4π Statistics 14 (Colin Rundel) Lecture 19 April, 1 16 / 19 f Rn (r) c n r n 1 ( 1 ) n e 1 r If we then perform a change of variables such that S n Rn / ds n f Sn (s) f Rn (r) dr n c n r n 1 e 1 r r c n r n e 1 r c n sn/ 1 e s/ Which is the Gamma distribution with k n/ and θ, S n Gamma(n/, ). This special case of the Gamma distribution is referred as a χ distribution with n-degrees of freedom. Statistics 14 (Colin Rundel) Lecture 19 April, 1 17 / 19 Finding c n Pearson s χ test If X Gamma(k, θ) then its density is given by f X (x k, θ) 1 1 θ k Γ(k) x k 1 e x/θ Based on this we then know that c c n 1 1 n/ Γ(n/) c n 1 n/ 1 Γ(n/) 1 c n () 1 n/ n/ 1 Γ(n/) () n/ c n n/ 1 Γ(n/) π n Γ(n/) Γ(1) c 3 3/ Γ(3/) 3 π 4π π/ Statistics 14 (Colin Rundel) Lecture 19 April, 1 18 / 19 Why do we care about the χ distribution? Imagine we want to decide if a six sided die is fair, we roll the die 1 times and total up the results Roll Counts In 19 Carl Pearson showed that k (O i E i ) k (N i np i ) χ (k 1) E i np i i1 i1 If you ve taken an introductory statistics course you ve seen this in the form of a hypothesis test where you calculate a p-value (probability of getting the observed value or more extreme given the H is true). p-value 1 F χ k 1 ( k i1 ) (O i E i ) Statistics 14 (Colin Rundel) Lecture 19 April, 1 19 / 19 E i