Lecture 6 Gamma distribution, χ -distribution, Student t-distribution, Fisher F -distribution. Gamma distribution. Let us tae two parameters α > and >. Gamma function is defined by If we divide both sides by we get α x α e x dx y α e y dy x α ( e x x α e x dx. where we made a change of variables x y. Therefore, if we define { α e x xα, x f(x α,, x < then f(x α, will be a probability density function since it is nonnegative and it integrates to one. Definition. The distribution with p.d.f. f(x α, is called Gamma distribution with parameters α and and it is denoted as Γ(α,. Next, let us recall some properties of gamma function. If we tae α > then using integration by parts we can write: x α e x dx x α d( e x ( e x (α x α dx (α x (α e x dx (α Γ(α. 35
Since for α we have we can write Γ( e x dx Γ(, Γ(3, Γ(4 3, Γ(5 4 3 and proceeding by induction we get that Γ(n (n! Let us compute the th moment of gamma distribution. We have, α α EX x x α e x dx x (α+ e x dx α Γ(α + α+ x α+ e x dx α+ Γ(α + }{{} p.d.f. of Γ(α +, integrates to α Γ(α + Γ(α + (α + Γ(α + α+ Therefore, the mean is the second moment is and the variance (α + (α +... α (α + α. EX then X +... + X n has distribution Γ(α +... + α n, Proof. If X Γ(α, then a moment generating function (m.g.f. of X is α α Ee tx e tx x α e x dx x α e ( tx dx α EX (α + α (α + α ( α α Var(X EX (EX. Below we will need the following property of Gamma distribution. Lemma. If we have a sequence of independent random variables X Γ(α,,..., X n Γ(α n, α ( t α x α e ( tx dx. ( t α }{{} 36
The function in the last (underbraced integral is a p.d.f. of gamma distribution Γ(α, t and, therefore, it integrates to. We get, ( α Ee tx. t Moment generating function of the sum n i X i is n n n P ( tx i αi ( αi t P n Ee i X i E e txi Ee t i i i and this is again a m.g.f. of Gamma distibution, which means that n ( n X i Γ α i,. i i t n-distribution. In the previous lecture we defined a n-distribution with n degrees of freedom as a distribution of the sum X +... + X n, where X i s are i.i.d. standard normal. We will now show that which n n-distribution coincides with a gamma distribution Γ(,, i.e. ( n n Γ,. Consider a standard normal random variable X N(,. Let us compute the distribution of X. The c.d.f. of X is given by P(X x P( x X x x x t α e dt. The p.d.f. can be computed by taing a derivative d P(X x and as a result the p.d.f. of dx X is x d f X (x x α e dt dx α e x x e x x α t ( ( α e. x ( x x α e ( x We see that this is p.d.f. of Gamma Distribution Γ(,, i.e. we proved that X Γ(,. Using Lemma above proves that X +... + X n Γ( n. Fisher F -distribution. Let us consider two independent random variables, ( ( m X Γ, and Y Γ,. m Definition: Distribution of the random variable X/ Z Y/m, 37
is called a Fisher distribution with degrees of freedom and m, is denoted by F,m. First of all, let us notice that since X can be represented as X +... + X for i.i.d. standard normal X,..., X, by law of large numbers, (X +... + X EX when. This means that when is large, the numerator X/ will concentrate near. Similarly, when m gets large, the denominator Y/m will concentrate near. This means that when both and m get large, the distribution F,m will concentrate near. Another property that is sometimes useful when using the tables of F -distribution is that ( F,m (c, F m,,. c This is because ( X/ ( Y/m ( F,m (c, P c P Y/m X/ F m,,. c c Next we will compute the p.d.f. of Z F,m. Let us first compute the p.d.f. of The p.d.f. of X and Y are X Z. m Y ( ( m y m e y f(x x e x and g(y Γ( Γ( m correspondingly, where x and y. To find the p.d.f of the ratio X/Y, let us first write its c.d.f. Since X and Y are always positive, their ratio is also positive and, therefore, for t we can write: ( X ( ty P t P(X ty f(xg(ydx dy Y since f(xg(y is the joint density of X, Y. Since we integrate over the set {x ty} the limits of integration for x vary from to ty. Since p.d.f. is the derivative of c.d.f., the p.d.f. of the ratio X/Y can be computed as follows: d ( X d ty dt P Y t f(xg(ydxdy f(tyg(yydy dt ( m m ( (ty e ty e y ydy Γ( Γ( m +m ( ( +m t e (t+y dy y y Γ( Γ( m }{{} 38
The function in the underbraced integral almost loos lie a p.d.f. of gamma distribution Γ(α, with parameters α ( + m/ and /, only the constant in front is missing. If we miltiply and divide by this constant, we will get that, ( +m ( +m d ( X P dt Y t + m Γ( Γ( m t Γ( ( t + y ( + m ( e (t+y +m dy +m ( t + Γ( Γ( + m t Γ( Γ( m ( + t +m, since the p.d.f. integrates to. To summarize, we proved that the p.d.f. of (/mz X/Y is given by Γ( + m f X/Y (t t ( + t +m. Γ( Γ( m Since ( X t ( t P(Z t P f Z (t P(Z t f X/Y Y m t m m, this proves that the p.d.f. of F,m -distribution is Γ( +m ( t t + m f,m (t +. Γ( Γ( m m m m ( Γ( + m / m m/ t (m + t +m. Γ( Γ( m Student t n -distribution. Let us recall that we defined t n -distibution as the distribution of a random variable X T (Y + + Y n n if X, Y,..., Y n are i.i.d. standard normal. Let us compute the p.d.f. of T. First, we can write, ( X P( t T t P(T t P (Y + + Y /n t. n If f T (x denotes the p.d.f. of T then the left hand side can be written as On the other hand, by definition, P( t T t X t t (Y +... + Y n /n f T (xdx. has Fisher F,n -distribution and, therefore, the right hand side can be written as t f,n (xdx. 39
We get that, t t f T (xdx f,n (xdx. t Taing derivative of both side with respect to t gives f T (t + f T ( t f,n (t t. But f T (t f T ( t since the distribution of T is obviously symmetric, because the numerator X has symmetric distribution N(,. This, finally, proves that Γ( n+ t f T (t f,n (t n+ t Γ( Γ( n n ( + n. 4