Continuous Random Variables, Moments and Moment Generating Function

Transcription

1 Continuous Random Variables, Moments and Moment Generating Function

2 Continuous random variable Definition: A random variable X is continuous if the CDF F X (x) = P(X x) is a continuous function of x. Example 1: Weight of a new born baby; Example 2: Waiting time in a bus stop.

3 Density function Definition: The probability density function f X (x) of a continuous random variable X is the function that satisfies F X (x) = x f X (t)dt. Remark: There exist continuous random variables which do not have densities. We will only discuss the case where the density exists.

4 CDF and density function If X has a density f X (x), then F X (x) = x f X (u)du. If F X (x) is differentiable, then d dx F X (x) x=x0 = F X (x 0) but F X (x 0) is not necessary to be the same as f X (x 0 ). If f X (x) is continuous at point x 0, then d dx F X (x) x=x0 = F X (x 0) = f X (x 0 ).

5 Example Define F X (x) = 2x 0 x < x x < 1 3 f X (x) = x 1 0 otherwise 1 3 x 1 fx (x) = 2 0 x < 1 3, x x = x 1 0 otherwise (a) F X (x) = x f X (t)dt = x f X (t)dt. (b) df X (x) dx x= 1 3 does not exist but f X ( 1 3 ) = 1 2. (c) df X (x) dx x= 1 6 = f X ( 1 6 ) f X ( 1 6 ).

6 Density is not probability Note that f X (t) is not probability. Actually, P(X = t) = 0 for any t if X is a continuous random variable. Because {X = t} {t ɛ < X t} for any ɛ P(X = t) P(t ɛ < X t) = F X (t) F X (t ɛ). Hence 0 P(X = t) lim ɛ 0 [F X (t) F X (t ɛ)] = 0 by continuity of F X. Any meaningful statement about probability must consider X lying in some interval. Probability is interpreted as the area under the density function.

7 Example: Logistic distribution A random variable X with logistic distribution if F X (x) = 1 1+e x. Then f X (x) = df X (x) dx = e x (1 + e x ) 2 P(a < X < b) = F X (b) F X (a) = b f X (x)dx a f X (x)dx = If x is small, P(a x a + x) f X (a) x. b a f X (x)dx.

8 Quantile and median Definition: Let X be a random variable with CDF F X (x). For any 0 < α < 1, an quantile of X is any number x α satisfying F X (x α ) α and F X (x α ) α. The median is x 0.5. Quantiles defined above may not be unique. To make it unique, we usually define the quantile as x α = inf{x : F X (x) α}.

9 Example Let X have CDF 0 u < 0 F(u) = u 2 /2 0 u < x < u u < 3 1 u 3. Where are 0.6, 0.65, 0,75 quantiles and where is the median?

10 Expected value Definition: Let X be a continuous random variable with density f (x). Then the expected value of a random variable g(x) is E(g(X)) = g(x)f (x)dx provided that g(x) f (x)dx exists. Linearity: E(ag 1 (X) + bg 2 (X) + c) = ae(g 1 (X)) + be(g 2 (X)) + c.

11 Examples Example 1: If X has exponential(λ), i.e., f X (x) = 1 λ exp( x ) x 0 and λ > 0. λ What is the expectation of X? Example 2: If X has Cauchy distribution, the density of X is f X (x) = 1 π What is the expectation of X? x 2 < x <.

12 Mixture of continuous and discrete random variables A random variable X could take continuous and discrete values. P(X A) = α x A f d (x) + (1 α) f c (x)dx A for any Borel set A and 0 < α < 1, where f d (x) is a pmf and f c (x) is a density. The expectation of X is E(X) = α x xf d (x) + (1 α) xf c (x)dx.

13 Example: Jelly Donut Problem Suppose that the Jelly Donut Company want to decide how many jelly donuts to bake every day. Package sells for s dollar and cost c dollars to make. The demand D is a continuous random variable with density f and CDF F. To maximize the profit, how many packages the company should make?

14 Variance For any random variable X, the variance is Var(X) = E[(X E(X)) 2 ]. Example: If X has exponential(λ), what is the variance of X?

15 Moments Definition: For each n, the nth moment of X is µ n = E(X n ) and the nth central moment of X is κ n = E(X µ) n where µ = E(X). The moments can be used to measure some aspects of a distribution. For example, we can measure degree of asymmetric of a distribution by coefficient of skewness: γ 1 = κ 3 σ 3 and for symmetric densities (pmfs), we can measure peakedness by coefficient of kurtosis γ 2 = κ 4 σ 4 3, where σ 2 = Var(X).

16 Symmetric A random variable is said to symmetric about 0 if X and X have the same distribution. If X and X have the same distribution, F(u) + F( u) = 1 + P(X = u). If X is continuous and f X (x) = f X ( x) for all x except countable many x, then X is symmetric about 0. If X is discrete, we require that f X (x) = f X ( x) for all x.

17 Left skew and right skew

18 Examples: Skewness What are the coefficients of skewness for exponential(λ) and Binomial(n, p)? Exponential Distribution Binomial Distribution(n=10) density λ=2 λ=5 probability p=0.1 p= x x

19 Kurtosis Kurtosis γ 2 = κ 4 σ 4 3 is compared with the kurtosis of the standard normal distribution, which has kurtosis 0.

20 Moments might not fully determine a distribution Moments reflects some aspects of a distribution, but it can not determine a distribution. Two totally different distributions could have all the moments to be the same. Example: Consider two random variables X and Y having densities f X (x) = 1 2πx exp( (log x) 2 /2) 0 < x < f Y (y) = f X (y)(1 + sin(2π log(y))) 0 < y <.

21 Example continuation Densities fy(y) fx(x) x

22 Moment generating function Let X be a random variable with CDF F X. The moment generating function (MGF) of X, denoted by M X (t) is M X (t) = E(e tx ) provided that the expectation exists for t in some neighborhood of 0. That is for all h < t < h, E(e tx ) exists. For continuous random variables with density function f X (x), M X (t) = e tx f X (x)dx. For discrete random variables with probability mass function p X (x), M X (t) = x etx p X (x).

23 Properties For any constants a and b, the MGF of the random variable ax + b is given by M ax+b (t) = e bt M X (at). For independent and identically distributed random variables X, X 1,, X n. Let S n = X 1 + X 2 + X X n. Then M Sn (t) = MX n (t), where M X (t) is the MGF of X.

24 Calculate moments using MGF If X has MGF M X (t), then E(X n ) = d n dt n M X (t) t=0. That is, the n-th moment is equal to the n-th derivatives of M X (t) evaluated at t = 0.

25 Examples (Discrete case) If X has Binomial(n, p), what is the moment generating function of X? (Continuous case) If X has Exponential(λ), what is the moment generating function of X?

26 MGF and CDF Let F X (x) and F Y (y) be two CDFs. If the moment generating functions exist and M X (t) = M Y (t) for all t in some neighborhood of 0, then F X (u) = F Y (u) for all u. Suppose {X n : n = 1, 2, } is a sequence of random variables, each with MGF M Xn. Further, lim M X n n (t) = M X (t) for all t in a neighborhood of 0 and M X (t) is MGF of X. Then there exist a unique CDF F X (x) whose moments are determined by M X (t) and for all x where F X (x) is continuous, we have lim F X n n (x) = F X (x).