October 3rd, Linear Algebra & Properties of the Covariance Matrix

Transcription

1 Linear Algebra & Properties of the Covariance Matrix October 3rd, 2012

2 Estimation of r and C Let rn 1, rn, t..., rn T be the historical return rates on the n th asset. rn 1 rṇ 2 r n =. r T n n = 1, 2,..., N.

3 Estimation of r and C The expected return is approximated by T ˆ r n = 1 T t=1 r t n and the covariance is approximated by ĉ mn = 1 T 1 or in matrix/vector notation Ĉ = 1 T 1 T (rn t ˆ r n )(rm t ˆ r m ), t=1 T (r t ˆ r)(r t ˆ r) an outer product. t=1

4 Importance of C The expected return r is often estimated exogenously (e.g. not statistically estimated). But statistical estimation of C is an important topic, is used in practice, and is the backbone of many finance strategies. A major part of Markowitz theory was the assumption for C to be a covariance matrix it must be symmetric positive definite (SPD).

5 Real Matrices/Vectors Let s focus on real matrices, and only use complex numbers if necessary. A vector x R N and a matrix A R M N are x 1 a 11 a a 1N x 2 x =. and A = a 21 a a 2N..... a M1 a M2... a MN x N

6 Matrix/Vector Multiplication The matrix A times a vector x is a new vector a 11 a a 1N x 1 a 21 a a 2N x 2 Ax = a M1 a M2... a MN x N a 11 x 1 + a 12 x a 1N x N a 21 x 1 + a 22 x a 2N x N =. RM. a M1 x 1 + a M2 x a MN x N Matrix/matrix multiplication is an extension of this.

7 Matrix/Vector transpose Their transposes are x 1 x x 2 =. x N a 11 a a 1N and A a 21 a a 2N =..... a M1 a M2... a MN Note that (Ax) = x A. = (x 1, x 2,..., x N ) a 11 a a M1 a 12 a a M2 = a 1N a 2N... a MN

8 Vector Inner/outer Product The vector x R N inner product with another vector y R N is x y = y x = x 1 y 1 + x 2 y x N y n. There is also the outer product, x 1 y 1 x 1 y 2... x 1 y N xy x 2 y 1 x 2 y 2... x 2 y N =..... yx. x N y 1 x N y 2... X N Y N

9 Matrix/Vector Norm The norm on the vector space R N is, and for any x R N we have x = x x. Properties of the norm x 0 for any x, x = 0 iff x = 0, x + y x + y (triangle inequality), x y x y with equality iff x = ay for some a R (Cauchy-Schwartz), x + y 2 = x 2 + y 2 if x y = 0 (Pythagorean thm).

10 Linear Independence & Orthogonality Two vectors x, y are linear independent if there is no scalar constant a R s.t. y = ax. These two vectors are orthogonal if x y = 0. Clearly, orthogonality linear independent.

11 Span A set of vectors x 1, x 2,..., x n is said to span a subspace V R N if for any y V there are constants a 1, a 2,..., a n such y = a 1 x 2 + a 2 x a n x n. We write, span(x 1,..., x n ) = V. In particular, a set x 1, x 2,..., x N will span R N iff they are linearly independent: span(x 1, x 2,..., x N ) = R N x i linearly independent of x j for all i, j. If so, then x 1, x 2,..., X N are a basis for R N.

12 Orthogonal Basis Definition A basis of R N is orthogonal if all its elements are orthogonal, span(x 1, x 2,..., x N ) = R N and x i x j = 0 i, j.

13 Invertibility Definition A square matrix A R N N is invertible if for any b R N there exists x s.t. Ax = b. If so, then x = A 1 b. A is invertible if any of the following hold: A has rows that are linearly independent A has columns that are linearly independent det(a) 0 there is no non-zero vector x s.t. Ax = 0. In fact, these 4 statements are equivalent.

14 Eigenvalues Let A be an N N matrix. A scalar λ C is an eigenvalue of A if Ax = λx for some vector x = re(x) + 1 im(x) C N. If so x is an eigenvector. By 4th statement of previous slide, A 1 exists zero is not an eigenvalue.

15 Eigenvalue Diagonalization Let A be an N N matrix with N linearly independent eigenvectors. There is an N N matrix X and an N N diagonal matrix Λ such that A = X ΛX 1, where Λ ii is an eigenvalue of A, and the columns of X = [x 1, x 2,..., x N ] are eigenvectors, Ax i = Λ ii x i.

16 Real Symmetric Matrices A matrix is symmetric if A = A. Proposition A symmetric matrix has real eigenvalues. pf: If Ax = λx then λ 2 x 2 = λ 2 x x = x A 2 x = x A Ax = Ax 2, which is real and nonnegative (x = re(x) 1 im(x) i.e the conjugate transpose). Hence, λ is cannot be imaginary or complex.

17 Real Symmetric Matrices Proposition Let A = A be a real matrix. A s eigenvectors form an orthogonal basis of R N, and the diagonalization is simplified: i.e. X 1 = X. A = X ΛX, Basic Idea of Proof: Suppose that Ax = λx and Ay = αy with α λ. Then λx y = x A y = x Ay = αx y, and so it must be that x y = 0.

18 Skew Symmetric Matrices A matrix is skew symmetric if A = A. Proposition A skew symmetric matrix has purely imaginary eigenvalues. pf: If Ax = λx then λ 2 x 2 = λ 2 x x = x A 2 x = x A Ax = Ax 2 which is less than zero. Hence, λ 2 < 0 must be purely imaginary.

19 Positive Definitness Definition A matrix A R N N is positive definite iff x Ax > 0 x R N. It is only positive semi-definite iff x Ax 0 x R N.

20 Positive Definitness Symmetric Proposition If A R N N is positive definite, then A = A, i.e. it is symmetric positive definite (SPD). pf: For any x R N we have 0 < x Ax = 1 x (A + A 2 } {{ } )x + x ( A A } {{ } )x, sym. skew-sym. but if A A 0 then there exists eigenvector x s.t. (A A )x = αx where α C \ R, which contradicts the inequality.

21 Eigenvalue of an SPD Matrix Proposition If A is SPD, then it has all positive eigenvalues. pf: By definition for any y R N \ {0} it holds that y Ay > 0. In particular, for any λ and x s.t. Ax = λx, we have λ x 2 = x Ax > 0 and so λ > 0. Consequence: for A SPD there s no x s.t. Ax = 0, and so A 1 exists.

22 So Can/How Do We Invert C? The invertibility of the covariance matrix is very important: we ve seen a need for it in the efficient frontiers and Markowitz problem, if it s not invertible then there are redundant assets, we ll need it to be invertible when we discuss multivariate Gaussian distributions. If there is no risk-free, our covariance matrix is practically invertible by definition: ( ) 0 < var w i r i = w i w j C ij = w Cw i i,j where w R N \ {0} is any allocation vector (w may not sum to 1). Hence, C is SPD C 1 exists.

23 So How Do We Know Ĉ is Covariance Matrix? Recall, Ĉ = 1 T 1 T t=1 (r t r)(r t r). Each summand is symmetric, (r t ˆ r)(r t ˆ r) is symmetric, and sum of symmetric matrices is symmetric. But each summand is not invertible because there exists a vector y s.t. y (r t ˆ r) = 0, which means that zero is an eigenvalue, (r t ˆ r) (r t ˆ r) y = 0. } {{ } =0

24 So How Do We Know Ĉ is Covariance Matrix? If span(r 1 ˆ r, r 2 ˆ r,..., r T ˆ r) = R N then Ĉ is invertible. So need T N. In order for Ĉ to be close C will need T N. Ĉ is also SPD: ( y Ĉy = y 1 T 1 ) T (r t ˆ r)(r t ˆ r) y t=1 = 1 T 1 T t=1 y (r t ˆ r)(r t ˆ r) y = 1 T 1 because at least 1 t s.t. y (r t ˆ r) 0. T (y (r t ˆ r)) 2 > 0, t=1

25 A Simple Covariance Matrix C = (1 ρ)i + ρu where I is the identity and U ij = 1 for all i, j. U1 = N1 Ux = 0 if x 1 = 0 where 1. = (1, 1,..., 1). Hence, for any x we have Cx = (1 ρ)ix + ρux = (1 ρ)x + ρ(1 x)1, which means x is an eigenvector iff 1 x = 0 or x = a1. The eigenvalues of C are 1 ρ + Nρ and 1 ρ, and the eigenvectors are 1 and the N 1 vectors orthogonal to 1, respectively.

26 Positive Definite (Complex Hermitian Matrices) Let C be a square matrix, i.e. C C N N. C is positive definite if z Cz > 0 z C N \ {0} where C N is N-dimensional complex vectors, and z is conjugate transpose, C is positive semi-definite if z = (z r + iz i ) = z r iz i. z Cz 0 z C N \ {0} If C positive definite and real, then it must also be symmetric i.e. C nm = C mn m, n.

27 Gershgorin Circles Proposition All eigenvalues of the matrix A lie in one of the Gershgorin circles, λ A ii j i A ij for at least 1 i N, where λ is an eigenvalue of A. pf: Let λ be an eigenvalue and let x be an eigenvector. Choose i so that x i = max j x j. We have j A ijx j = λx j. W.L.O.G. we can take x i = 1 so that j i A ijx j = λ A ii, and taking absolute values yields the result, λ A ii = A ij x j j i A ij x j A ij. j i j i