Economics 102C: Advanced Topics in Econometrics 2 - Tooling Up: The Basics

Transcription

1 Economics 102C: Advanced Topics in Econometrics 2 - Tooling Up: The Basics Michael Best Spring 2015

2 Outline The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices

3 The Evaluation Problem: Why Do We Do Microeconometrics? Microeconometrics = statistical tools to establish causality prediction Answer economic policy questions such as: Do job training programs help participants find jobs and earn higher wages? How much more do people earn as a result of going to college? Do higher minimum wages increase unemployment? Do higher taxes make people work less? Using data, but informed by theory The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 2 / 47

4 The Selection Problem: Why This Isn t Trivial Example 1: Do hospitals make people healthier? Would you say your health in general is excellent (5), very good (4), good (3), fair (2), poor (1)? Group Sample Size Mean Health Status Std. Error Hospital 7, No Hospital 90, Source: National Halth Interview Survey (NHIS) 2005, via Mostly Harmless Econometrics Difference in means is 0.72 (t=58.9) So people who go to hospital feel worse? The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 3 / 47

5 The Selection Problem: Why This Isn t Trivial Example 2: Do higher tax rates make people work less? Source: Feldstein (1995) So higher taxes make people richer? The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 4 / 47

6 The Selection Problem: Why This Isn t Trivial Example 3: Do more police officers reduce crime rates? Source: Levitt (1997) So more police increase crime rates? The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 5 / 47

7 Potential Outcomes: A Framework for Thinking About This In Define a binary random variable, D i = {0, 1}: The treatment variable. Outcome of interest y i (health status) Does D i affect y i? Potential outcome Hospitals example: Potential outcome = { y 1i if D i = 1 y 0i if D i = 0 y1i is the health status i would report if they go to hospital, irrespective of whether they actually go y0i is the health status i would report if they don t go to hospital, irrespective of whether they actually go The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 6 / 47

8 Potential Outcomes: Causal Effects and Observed Outcomes Causal effect of going to hospital on person i is y 1i y 0i Unfortunately, we never observe both y 1i and y 0i. Instead we observe { y 1i if D i = 1 y i = y 0i if D i = 0 = y 0i + (y 1i y 0i ) D i y 1i y 0i can be very different for different people. But we can learn about averages The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 7 / 47

9 Detour: Expectations Recall the expectation operator E [y] = ˆ Y f y (Y ) dy and the conditional expectation operator E [y x = X] = = ˆ ˆ Y f y (Y x = X) dy Y f x,y (Y, X) dy f x (X) The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 8 / 47

10 Potential Outcomes: Average Treatment Effects Return to table 1, we observe: E [yi D i = 1] = 3.21 E[y i D i = 0] = 3.93 E [y i D i = 1] E [y i D i = 0] = 0.72 = E [y 1i D i = 1] E [y 0i D i = 1]?? Decompose the observed difference: E [y i D i = 1] E [y i D i = 0] }{{} Observed difference in average health = E [y 1i D i = 1] E [y 0i D i = 1] }{{} Average treatment effect on the treated + E [y 0i D i = 1] E [y 0i D i = 0] }{{} Selection bias We want to estimate ATT = E [y 1i y 0i D i = 1] but need to deal with selection bias: Enter microeconometrics The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 9 / 47

11 What Distinguishes Microeconometrics? Data is at the level of the individual, firm or group (not economy-wide aggregates) Panel data common Deal with discrete and/or censored data use non-linear methods Deal with heterogeneity explicitly The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 10 / 47

12 Randomization and The Selection Problem Randomized Control Trials (RCTs) are becoming increasingly prevalent in applied economics: [AEA RCT Registry] Randomly assign some units to treatment (D i = 1) and others to control (D i = 0) Implies that D i is independent of y 1i and y 0i E [y i D i = 1] E [y i D i = 0] = E [y 1i D i = 1] E [y 0i D i = 0] Moreover, we can simplify further = E [y 1i D i = 1] E [y 0i D i = 1] independence of D i, y 1i &y 0i = E [y 1i y 0i D i = 1] = AT T E [y 1i y 0i D i = 1] = E [y 1i y 0i ] = AT E The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 11 / 47

13 Outline The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices

14 Matrices A matrix is a rectangular array of numbers. An r c matrix has r rows and c columns a 11 a 12 a 13 a 1c a 21 a 22 a 23 a 2c A = [a ij ] = a 31 a 32 a 33 a 3c a r1 a r2 a r3 a rc A matrix is square if r = c The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 13 / 47

15 Vectors A row vector is a 1 c matrix x = (x 1, x 2, x 3,..., x c ) A column vector is an r 1 matrix y 1 y 2 y = y 3. y r The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 14 / 47

16 Square Matrices A square matrix is symmetric if a ij = a ji i, j The main or principal diagonal of a square matrix is diag (A) = (a 11, a 22, a 33,..., a cc ) A square matrix is diagonal if a ij = 0 i j A = a a a a cc The n n identity matrix I n is a square, diagonal matrix with diag (I n ) = (1, 1, 1,..., 1) The trace of a matrix is the sum of its diagonal elements tr (A) = n i=1 a ii The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 15 / 47

17 Matrix Operations (arithmetic) A = B if a ij = b ij i, j C = A + B = [a ij + b ij ] C = a 11 + b 11 a 12 + b 12 a 13 + b 13 a 1c + b 1c a 21 + b 21 a 22 + b 22 a 23 + b 23 a 2c + b 2c a 31 + b 31 a 32 + b 32 a 33 + b 33 a 3c + b 3c a r1 + b r1 a r2 + b r2 a r3 + b r3 a rc + b rc Requires both A and B to have the same dimensions The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 16 / 47

18 Transposing Matrices The transpose of a matrix flips the rows and columns of the matrix We denote the transpose of A by A [a ji ] If A is r c then A is c r If A is symmetric, then A = A By definition (A ) = A If a is a column vector, then a is a row vector (A + B) = A + B The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 17 / 47

19 Multiplying Matrices To multiply two matrices A n m and B m p, the number of columns in A must equal the number of rows in B. Multiplying gives AB = [ ab ij = p k=1 a ik b kj ] That is, the ij th element of A is the product of the i th row of A and the j th column of B: b 1j b 2j (a i1, a i2,, a im ) b 3j p = a ik b kj = ab ij. k=1 In general, AB BA b mj The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 18 / 47

20 Matrix Products: Practice Turn to your neighbor: Consider the matrices P = I = Find IP 2. Find (P + I) and verify that it is equal to P + I 3. Find P P The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 19 / 47

21 Matrix Products: Practice-Solutions 1. Find IP: IP = = = (1 1) + (0 1) + (0 1) (1 2) + (0 3) + (0 0) (1 1) + (0 2) + (0 4) (0 1) + (1 1) + (0 1) (0 2) + (1 3) + (0 0) (0 1) + (1 2) + (0 4) (0 1) + (0 1) + (1 1) (0 2) + (0 3) + (1 4) (0 1) + (0 2) + (1 4) = P The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 20 / 47

22 Matrix Products: Practice-Solutions 2. Find (P + I) and verify that it is equal to P + I : (P + I) = = = (P + I) = P = I = I P + I = = (P + I) The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 21 / 47

23 Matrix Products: Practice-Solutions 3 find P P: P P = = = (1 1) + ( 1 1) + (1 1) (1 2) + ( 1 3) + (1 0) (1 1) + ( 1 2) + (1 4) (2 1) + (3 1) + (0 1) (2 2) + (3 3) + (0 0) (2 1) + (3 2) + (0 4) (1 1) + (2 1) + (4 1) (1 2) + (2 3) + (4 0) (1 1) + (2 2) + (4 4) The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 22 / 47

24 Properties of Matrix Products IA = A and AI = A where I are appropriate identity matrices (AB) = B A ABC = (AB)C = A(BC) A (B + C) = AB + AC; (A + B) C = AC + BC for A n m, A A m m and AA n n are symmetric, but not equal. For example, for x n 1 x x 1 1 = (x 1, x 2,, x n ) xx n n = x 1 x 2. x n x 1 x 2. x n = n i=1 (x 1, x 2,, x n ) = x 2 i x 2 1 x 1 x 2 x 1 x n x 2 x 1 x 2 2 x 2 x n x n x 1 x n x 2 x 2 n The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 23 / 47

25 Properties of Matrix Products If A n n is diagonal, then x 1 n Ax n 1 = n i=1 a iix 2 i is a weighted sum of squares tr (A A) = tr (AA ) a (square) matrix B is idempotent if B = B 2 If B is symmetric and idempotent, then B B = B The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 24 / 47

26 The Identity Vector and the Centering Matrix the identity vector is a column vector of ones 1 1 ι n 1 =. 1 so ι ι = n and ι x n 1 = x n 1 ι = n i=1 x i and (ι ι) 1 ι x = 1 n n i=1 x i = x (ι ι) 1 ιι is an n n matrix with all elements equal to 1/n Since x = Ix, x 1 x x 2 x. x n x [ = x ( ι ι ) ] 1 ιι x where M 0 is the centering matrix = [ I ( ι ι ) 1 ιι ] x = M 0 x The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 25 / 47

27 The Centering Matrix M 0 has diagonal elements (1 1/n) and off diagonal elements 1/n so it is symmetric and idempotent We also find that n (x i x) 2 = ( M 0 x ) ( M 0 x ) = x M 0 M 0 x = x M 0 x i=1 And we can combine 2 vectors x and y to arrive at [ n i=1 (x i x) 2 n i=1 (x ] i x) (y i ȳ) n i=1 (y i ȳ) (x i x) n i=1 (y i ȳ) 2 [ x = M 0 x x M 0 ] y y M 0 x y M 0 y The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 26 / 47

28 Matrix Rank The column rank of a matrix is the maximum number of linearly independent columns of the matrix. A column is linearly independent of the other columns if we can t write it as a linear combination of the other columns e.g if a = 1 2 2, b = 1 0 4, c = then c = 2a + b and so c is not linearly independent of a and b Similarly, the row rank of a matrix is the maximum number of linearly independent rows of the matrix. It turns out the row and column rank of a matrix are equal, so we just talk about the rank of a matrix A: rank (A) The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 27 / 47

29 Matrix Rank For a matrix A r c rank (A) min {r, c} If rank (A) = min {r, c} then A has full rank. If rank (A) < min {r, c} then A is rank deficient Furthermore (don t prove) rank (A) = rank (A A) = rank (AA ) The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 28 / 47

30 Matrix Determinant The minor A ij of a matrix A is the matrix you get by deleting the i th row and the j th column from A ( ) ( ) e.g. A = , A 11 = & A = The co-factor of a matrix minor A ij is defined as c ij = ( 1) i+j A ij where A is the determinant of a matrix, defined as... The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 29 / 47

31 Matrix Determinant The determinant of a matrix A n n is A = n a ij c ij = j=1 n a ij ( 1) i+j A ij for any row i (or column) of the matrix A ( ) a b Note that for a 2 2 matrix A =, A c d 11 = d, A 12 = c, A 21 = b and A 22 = a. A = ad bc j=1 Note that the determinant of a matrix is a scalar The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 30 / 47

32 Matrix Determinant: Practice Turn to your neighbor: Together, find the determinant of the matrix P = The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 31 / 47

33 Matrix Determinant: Practice-Solutions Using Row 1 of P: ( 3 2 P 11 = 0 4 ) ( 1 2 P 12 = 1 4 ) ( 1 3 P 13 = 1 0 P 11 = (3 4) (2 0) = 12 P 12 = ( 1 4) (2 1) = 6 P 13 = ( 1 0) (3 1) = 3 c 11 = ( 1) 1+1 P 11 = 12 c 12 = ( 1) 1+2 P 12 = 6 c 13 = ( 1) 1+3 P 13 = 3 P = (1 12) + (2 6) + (1 3) = 21 ) Using row 3 of P : ( 2 1 P 31 = 3 2 ) ( 1 1 P 32 = 1 2 ) ( 1 2 P 33 = 1 3 P 31 = (2 2) (1 3) = 1 P 32 = (1 2) (1 1) = 3 P 33 = (1 3) (2 1) = 5 c 31 = ( 1) 3+1 P 31 = 1 c 32 = ( 1) 3+2 P 32 = 3 c 33 = ( 1) 3+3 P 33 = 5 P = (1 1) + (0 3) + (4 5) = 21 ) The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 32 / 47

34 Inverse of a Matrix Think of the inverse as a bit like dividing An n n matrix A has an inverse, denoted by A 1 if and only if A 1 A = I n and AA 1 = I n Think of the scalar case: a a 1 = 1 a 1 = 1 a However, for matrices it s usually not true that a 1 ij = 1 a ij A matrix that has no inverse is singular, or non-invertible Turns out that for an n n matrix A A is singular A = 0 rank (A) < n The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 33 / 47

35 Properties of the Matrix Inverse If a matrix has an inverse, it is unique (αa) 1 = (1/α) A 1 for scalar α 0 and A 1 ( exists A 1) 1 = A (A ) 1 = ( A 1) (AB) 1 = B 1 A 1 The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 34 / 47

36 Outline The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices

37 Matrix Calculus Recall, that if x is a scalar and we have a function y = f (x) g (x) = f (x) x f (x) x = = y x When, instead y = f (x 1, x 2,..., x n ) = f (x), the vector of partial derivatives, the gradient vector is y/ x 1 y/ x 2. y/ x n The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 36 / 47

38 Matrix Calculus The second derivatives matrix, or the Hessian matrix H = 2 y/ x y/ x 1 x 2 2 y/ x 1 x n 2 y/ x 2 x 1 2 / x y/ x 2 x n y/ x n x 1 2 y/ x n x 2 2 y/ x 2 n is a square, symmetric matrix. Note that [ ( y/ x) ( y/ x) H = ( y/ x) ] x 1 x 2 x n ( y/ x) = (x 1, x 2,..., x n ) = ( y/ x) x = 2 y x x The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 37 / 47

39 Matrix Calculus In particular, consider a linear function y = f (x 1, x 2,..., x n ) = a 1 x 1 + a 2 x a n x n which we can write as y = a x = x a = n a i x i i=1 Then we have that g (x) = a x x =a The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 38 / 47

40 Matrix Calculus What about differentiating a vector wrt another vector? Consider the set of linear functions y = Ax so that y i = a i x where a i is the ith row of A Then y i / x = a i, the transpose of the i th row of A This means that y 1 / x y 2 / x. y n / x = [a 1, a 2,..., a n ] Ax x = A The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 39 / 47

41 Quadratic Forms A quadratic form is a way of writing a quadratic polynomial n n x Ax = x i x j a ij For example if A = i=1 j=1 [ ] then (x Ax) x In general, x Ax = 1x x x 1 x 2 [ ] 2x1 + 6x = 2 = 6x 1 + 8x 2 (x Ax) x [ = ( A + A ) x ] [ x1 x 2 ] = 2Ax The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 40 / 47

42 Matrix Calculus: Practice Turn to your neighbor and together consider the matrices x 1 P = x = x x 3 1. Derive Px x and verify that it is equal to P 2. Derive x Px x and verify that it is equal to (P + P ) x The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 41 / 47

43 Matrix Calculus: Practice-Solutions 1. Derive Px x Px = x 1 x 2 x 3 = x 1 + 2x 2 + x 3 x 1 + 3x 2 + 2x 3 x 1 + 4x 3 ( x +3x +2x ) (x +4x ) (x 1 +2x 2 +x 3 ) Px x 1 x 1 x 1 x = (x 1 +2x 2 +x 3 ) ( x 1 +3x 2 +2x 3 ) (x 1 +4x 3 ) x 2 x 2 x 2 (x 1 +2x 2 +x 3 ) ( x 1 +3x 2 +2x 3 ) (x 1 +4x 3 ) x 3 x 3 x 3 = = P The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 42 / 47

44 2. Derive x Px x Matrix Calculus: Practice-Solutions x (Px) = ( x 1 x 2 x 3 ) x 1 + 2x 2 + x 3 x 1 + 3x 2 + 2x 3 x 1 + 4x 3 = x 1 (x 1 + 2x 2 + x 3 ) + x 2 ( x 1 + 3x 2 + 2x 3 ) + x 3 (x 1 + 4x 3 ) = x x 1x 2 + 2x 1 x 3 + 3x x 2x 3 + 4x 2 3 x Px = x x 2 1 +x 1 x 2 +2x 1 x 3 +3x2 2 +2x 2 x 3 +4x2 3 x 1 x 2 1 +x 1 x 2 +2x 1 x 3 +3x2 2 +2x 2 x 3 +4x2 3 x 2 x 2 1 +x 1 x 2 +2x 1 x 3 +3x2 2 +2x 2 x 3 +4x2 3 x 3 = 2x 1 + x 2 + 2x 3 x 1 + 6x 2 + 2x 3 2x 1 + 2x 2 + 8x 3 ( = P + P ) x = x 1 x 2 x 3 The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 43 / 47

45 Outline The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices

46 OLS With Matrices Consider the OLS model for N individuals and k < N regressors. We have y 1 = β 1 + β 2 x 12 + β 3 x β k x 1k + u 1 y 2 = β 1 + β 2 x 22 + β 3 x β k x 2k + u 1. =. y N = β 1 + β 2 x 1N + β 3 x 1N β k x 1N + u N or, in matrices: y 1 1 x 12 x 13 x 1k y 2. = 1 x 22 x 23 x 2k y N 1 x N2 x N3 x Nk y = X β + u N 1 N k k 1 N 1 β 1 β 2. β k + u 1 u 2. u N The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 45 / 47

47 The OLS problem is: min β f (β) = OLS With Matrices N u 2 i = i=1 N ( yi x iβ ) 2 i=1 = û û = (y Xβ) (y Xβ) = y y y Xβ β X y + β X Xβ Differentiate to find the optimum ˆβ: f (β) / β = 0: 0 = y X ˆβ β ˆβ X y β = X y X y + 2X X ˆβ = 2X y + 2X X ˆβ + ˆβ X X ˆβ β The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 46 / 47

48 OLS With Matrices Need to solve X Xˆβ =X y If (X X) is invertible then we can pre-multiply both sides by (X X) 1 to get ˆβ = ( X X ) 1 X y When is (X X) invertible? Need (X X) to have full rank: No multicollinearity rank ( X X ) = rank (X) = k Is the second order condition satisfied? The Evaluation Problem Matrix Algebra Matrix Calculus OLS With Matrices 47 / 47