Expected Value Let X be a discrete random variable which takes values in S X = {x 1, x 2,..., x n } Expected Value or Mean of X: E(X) = n x i p(x i ) i=1 Example: Roll one die Let X be outcome of rolling one die. The frequency function is p(x) = 1, x = 1,..., 6, 6 and hence E(X) = 6 x=1 x 6 = 7 2 = 3.5 Example: Bernoulli random variable Let X Bin(1, θ). p(x) = θ x (1 θ) 1 x Thus the mean of X is E(X) = 0 (1 θ) + 1 θ = θ. Expected Value and Variance, Feb 2, 2003-1 -
Expected Value Linearity of the expected value Let X and Y be two discrete random variables. Then E(a X + b Y ) = ae(x) + be(y ) for any constants a, b R Note: No independence is required. Proof: E(a X + b Y ) = x,y(a x + b y)p(x, y) p(x, y) = p(y) x = a x,y = a x x p(x, y) + b y p(x, y) x,y x p(x) + b y y p(y) = ae(x) + be(y ) Example: Binomial distribution Let X Bin(n, θ). Then X = X 1 +...+X n with X i Bin(1, θ): E(X) = n E(X i ) = n θ = nθ i=1 i=1 Expected Value and Variance, Feb 2, 2003-2 -
Example: Poisson distribution Expected Value Let X be a Poisson random variable with parameter λ. E(X) = Remarks: x=0 x λx x! e λ = λ e λ = λ e λ e λ = λ x=0 λ x 1 (x 1)! For most distributions some advanced knowledge of calculus is required to find the mean. Use tables for means of commonly used distribution. Expected Value and Variance, Feb 2, 2003-3 -
Expected Value Example: European Call Options Agreement that gives an investor the right (but not the obligation) to buy a stock, bond, commodity, or other instruments at a specific time at a specific price. What is a fair price P for European call options? If S T is the price of the stock at time T, the profit will be Profit = (S T K) + P. Profit is a random variable. 0 10 0 10 20 30 0 10 20 30 40 50 0 Fair price P for this option is expected value P = E(S T K) +. Expected Value and Variance, Feb 2, 2003-4 -
Expected Value Example: European Call Options (contd) Consider the following simple model: S t = S t 1 + ε t, t = 1,..., T P(ε t = 1) = p and P(ε t = 1) = 1 p. S t is also called a random walk. The distribution of S T is given by (s 0 known at time 0) S T = s 0 + 2 Y T, with Y Bin(T, p) Therefore the price P is (assuming s 0 = 0 without loss of generality) P = E(S T K) + = T (2 y T K) p θ (y) 1 {y>(k+t )/2} y=1 Let n = 20, K = 10, θ = 0.6 P = 2.75 0.5 0.4 0.3 2.75 0.75 1.25 3.25 5.25 7.25 9.25 11.25 13.25 15.25 17.25 19.25 p(x) 0.2 0.1 0.0 Profit Frequency function of profit Expected Value and Variance, Feb 2, 2003-5 -
Example: Group testing Expected Value Suppose that a large number of blood samples are to be screened for a rare disease with prevalence 1 p. If each sample is assayed individually, n tests will be required. Alternative scheme: n samples, m groups with k samples Split each sample in half and pool all samples in one group Test pooled sample for each group If test positive test all samples in group separately What is the expected number of tests under this alternative scheme? Let X i be the number of tests in group i. The frequency function of X i is { p k if x = 1 p(x) = 1 p k if x = k + 1 The expected number of tests in each group is E(X i ) = p k + (k + 1)(1 p k ) = k + 1 kp k Hence E(N) = m Plot of E(N): i=1 E(X i ) = n (1 + 1 ) k pk The mean is minimized for groups of size 11. Proportion 0.20 0.25 0.30 0.35 0.40 0.45 0.50 2 4 6 8 10 12 14 16 k Expected Value and Variance, Feb 2, 2003-6 -
Let X be a random variable. Variance of X: var(x) = E ( X E(X) ) 2. Variance The variance of X is the expected squared distance of X from its mean. Suppose X is discrete random variable with S X = {x 1,..., x n }. Then the variance of X can be written as var(x) = n (x i n 2 x j p(x j )) p(xi ) i=1 j=1 Example: Roll one die X takes values in {1, 2, 3, 4, 5, 6} with frequency function p(x) = 1 6. E(X) = 6 x 1 x=1 6 = 7 2 var(x) = 6 ( 7 x 2 x=1 ) 2 1 6 = 1 6 ( 25 4 + 9 4 + 1 4 + 1 4 + 9 4 + 25 ) = 35 4 12 We often denote the variance of a random variable X by σ 2 X, σ 2 X = var(x) and its standard deviation by σ X. Expected Value and Variance, Feb 2, 2003-7 -
Properties of the Variance The variance can also be written as var(x) = E(X 2 ) ( E(X) ) 2 To see this (using linearity of the mean): var(x) = E(X E(X)) 2 = E [ X 2 2XE(X) + ( E(X) ) 2] = E ( X 2) 2E(X)E(X) + ( E(X) ) 2 = E(X 2 ) ( E(X) ) 2 Example: Let X Bin(1, θ). Then var(x) = E(X 2 ) ( E(X) ) 2 = E(X) ( E(X) ) 2 = θ θ 2 = θ (1 θ) Rules for the variance: For constants a and b var(ax + b) = a 2 var(x). For independent random variables X and Y var(x + Y ) = var(x) + var(y ). Example: Let X Bin(n, θ). Then var(x) = n θ (1 θ) Expected Value and Variance, Feb 2, 2003-8 -
Covariance For independent random variables X and Y we have var(x + Y ) = var(x) + var(y ). Question: What about dependent random variables? It can be shown that var(x + Y ) = var(x) + var(y ) + 2 cov(x, Y ) where cov(x, Y ) = E [ (X E(X))(Y E(Y ) ] is the covariance of X and Y. Properties of the covariance cov(x, Y ) = E(XY ) E(X) E(Y ) cov(x, X) = var(x) cov(x, 1) = 0 cov(x, Y ) = cov(y, X) cov(a X 1 + b X 2, Y ) = a cov(x 1, Y ) + b cov(x 2, Y ) Expected Value and Variance, Feb 2, 2003-9 -
Important: Covariance cov(x, Y ) = 0 does NOT imply that X and Y are independent. Example: Suppose X { 1, 0, 1} with probabilities P(X = x) = 1 3 for x = 1, 0, 1. Then E(X) = 0 and cov(x, X 2 ) = E(X 3 ) = E(X) = 0 On the other hand P(X = 1, X 2 = 0) = 0 1 9 = P(X = 1)P(X2 = 0), that is, X and Y are not independent! Note: The covariance of X and Y measures only linear dependence. Expected Value and Variance, Feb 2, 2003-10 -
Correlation The correlation coefficient ρ is defined as ρ XY = corr(x, Y ) = cov(x, Y ) var(x)var(y ). Properties: dimensionless quantity not affected by linear transformations, i.e. corr(a X + b, c Y + d) = corr(x, Y ) 1 ρ XY 1 ρ XY = 1 if and only if P(Y = a + b X) = 1 for some a and b measures linear association between X and Y Example: Three boxes: pp, pd, and dd (Ex 3.6) Let X i = 1 {penny on ith draw}. Then X i Bin(1, p) with p = 1 2 and joint frequency function p(x 1, x 2 ): x 1 \x 2 0 1 1 0 3 1 1 6 1 6 1 3 Thus: cov(x 1, X 2 ) = E[(X 1 p)(x 2 p)] = 1 4 1 3 + 1 4 1 3 + 2 1 4 1 6 = 1 12 corr(x 1, X 2 ) = 4 1 1 12 = 1 3 Expected Value and Variance, Feb 2, 2003-11 -
Prediction An instructor standardizes his midterm and final so the class average is µ = 75 and the SD is σ = 10 on both tests. The correlation between the tests is always around ρ = 0.50. X - score of student on the first examination Y - score of student on the second examination Since X and Y are dependent we should be able to predict the score in the final from the midterm score. Approach: Predict Y from linear function a + b X Minimize mean squared error Solution: MSE = E ( Y a b X ) 2 = var(y b X) + [ E(Y a b X) ] 2 a = µ b µ and b = σ XY σ 2 X = ρ Thus the best linear predictor is Ŷ = µ + ρ (X µ) Note: We expect the student s score on the final to differ from the mean only by half the difference observed in the midterm (regression to the mean). Expected Value and Variance, Feb 2, 2003-12 -
Summary Bernoulli distribution - Bin(1, θ) p(x) = θ x (1 θ) 1 x Binomial distribution - Bin(n, θ) ( ) n p(x) = θ x (1 θ) n x x E(X) = θ var(x) = θ(1 θ) E(X) = nθ var(x) = nθ(1 θ) Poisson distribution - Poiss(λ) p(x) = λx x! e λ E(X) = λ Geometric distribution p(x) = θ(1 θ) x 1 Hypergeometric distribution - H(N, M, n) p(x) = var(x) = λ E(X) = 1 θ var(x) = 1 θ θ 2 ( M )( N M ) x n x ) E(X) = n M N ( N n Expected Value and Variance, Feb 2, 2003-13 -