Advanced Quantitative Methods: Mathematics (p)review

Transcription

1 Advanced Quantitative Methods: Mathematics (p)review University College Dublin 22 January 2013

2 1 Course outline 2 3

3 Outline Course outline 1 Course outline 2 3

4 Introduction Course outline Topic: advanced quantitative methods in political science.

5 Introduction Course outline Topic: advanced quantitative methods in political science. Or, alternatively: basic econometrics, as applied in political science.

6 Introduction Course outline Topic: advanced quantitative methods in political science. Or, alternatively: basic econometrics, as applied in political science. Or, alternatively: linear regression and some non-linear extensions.

7 Readings Course outline Peter Kennedy (2008), A guide to econometrics. 6th ed., Malden: Blackwell. Damodar N. Gujarati (2009), Basic econometrics. 5th ed., Boston: McGraw-Hill. Julian J. Faraway (2005), Linear models with R. Baco Raton: Chapman & Hill.

8 Readings Course outline Peter Kennedy (2008), A guide to econometrics. 6th ed., Malden: Blackwell. Damodar N. Gujarati (2009), Basic econometrics. 5th ed., Boston: McGraw-Hill. Julian J. Faraway (2005), Linear models with R. Baco Raton: Chapman & Hill. Andrew Gelman & Jennifer Hill (2007), Data analysis using regression and multilevel/hierarchical models. Cambridge: Cambridge University Press. William H. Greene (2003), Econometric analysis. 5th ed., Upper Saddle River: Prentice Hall.

9 Topics Course outline 1 22/1 Mathematics review 2 29/1 Statistical estimators 3 5/2 Ordinary Least Squares 4 12/2 Hypothesis testing 5 19/2 Regression diagnostics 6 26/2 Time-series analysis 7 5/3 Causal inference Study break 8 26/3 Maximum Likelihood 9 2/4 Limited dependent variables I 10 9/4 Limited dependent variables II 11 16/4 Bootstrap and simulation 12 23/4 Multilevel data

10 Frequentist vs Bayesian Frequentist statistics interprets probability as the frequency of occurrance in (hypothetically) many repetitions. E.g. if we throw this dice infinitely many times, what proportion of times would it be heads? We can here also talk of conditional probabilities: what would this frequency be if... and some condition follows.

11 Frequentist vs Bayesian Frequentist statistics interprets probability as the frequency of occurrance in (hypothetically) many repetitions. E.g. if we throw this dice infinitely many times, what proportion of times would it be heads? We can here also talk of conditional probabilities: what would this frequency be if... and some condition follows. Bayesian statistics interprets probability as a belief: if I throw this dice, what do you think is the chance of getting heads? We can now talk of conditional probabilities in a different way: how would your belief change given that... and some condition follows.

12 Frequentist vs Bayesian Frequentist statistics interprets probability as the frequency of occurrance in (hypothetically) many repetitions. E.g. if we throw this dice infinitely many times, what proportion of times would it be heads? We can here also talk of conditional probabilities: what would this frequency be if... and some condition follows. Bayesian statistics interprets probability as a belief: if I throw this dice, what do you think is the chance of getting heads? We can now talk of conditional probabilities in a different way: how would your belief change given that... and some condition follows. This course is a course in the frequentist analysis of regression models.

13 Homeworks Course outline 50 % Five homeworks due at specified dates 50 % Replication paper due May 5, 2012, 5 pm No exam

14 Homeworks Course outline 50 % Five homeworks due at specified dates 50 % Replication paper due May 5, 2012, 5 pm No exam Working with others is a good idea

15 Grade conversions Homeworks UCD TCD Homeworks UCD TCD % A+ A % E+ D 91-94% A A 37-44% E D 87-90% A- A 33-36% E- D 83-86% B+ B+ 0-32% F F 79-82% B B 75-78% B- B 70-74% C+ C % C C 62-65% C C 58-61% D+ C 54-57% D C 50-53% D- C

16 Replication paper Find replicable paper now and check whether appropriate Contact authors asap if you need their data See Gary King, Publication, publication

17 Syllabus and website Website: Syllabus downloadable there Slides and notes on website Data for exercises on website Booklet with commands available

18 Outline Course outline 1 Course outline 2 3

19 Vectors: examples 3 v = 4 1 w = [ ] 5

20 Vectors: examples 3 v = 4 1 w = [ ] 5 β 0 β 1 β = β 2 β 3 β 4

21 Vectors: examples 3 v = 4 1 w = [ ] 5 β 0 β 1 β = β 2 β 3 β 4 v and β are column vectors, while w is a row vector. When not specified, assume a column vector.

22 Vectors: transpose 3 v = 4 1 v = [ ] 5

23 Vectors: summation 3 5 v = N v i = i = 21

24 Vectors: addition =

25 Vectors: multiplication with scalar 2 v = v =

26 Vectors: inner product 5 7 v = 3 1 w = v w = = 64

27 Vectors: outer product 5 7 v = 3 1 w = vw =

28 Matrices: examples M = I =

29 Matrices: examples M = I = The latter is called an identity matrix and is a special type of diagonal matrix. Both are square matrices.

30 Matrices: transpose X = X =

31 Matrices: transpose X = X = (A ) = A

32 Matrices: indexing x 11 x 12 x 13 X 4 3 = x 21 x 22 x 23 x 31 x 32 x 33 x 41 x 42 x 43

33 Matrices: indexing x 11 x 12 x 13 X 4 3 = x 21 x 22 x 23 x 31 x 32 x 33 x 41 x 42 x 43 Always rows first, columns second.

34 Matrices: indexing x 11 x 12 x 13 X 4 3 = x 21 x 22 x 23 x 31 x 32 x 33 x 41 x 42 x 43 Always rows first, columns second. So X n k is a matrix with n rows and k columns. y n 1 is a column vector with n elements.

35 Matrices: addition =

36 Matrices: addition = (A+B) = A +B

37 Matrices: multiplication with scalar X = X =

38 Matrices: multiplication [ ] [ ] A = B = [ ] AB = = [ ]

39 Matrices: multiplication [ ] [ ] A = B = [ ] AB = = [ ] (ABC) = C B A (AB)C = A(BC) A(B+C) = AB+AC (A+B)C = AC+BC

40 Special matrices Symmetric matrix A = A Idempotent matrix A 2 = A Positive-definite matrix x Ax > 0 x 0 Positive-semidefinite matrix x Ax 0 x 0

41 Matrix rank Course outline The rank of a matrix is the maximum number of independent columns or rows in the matrix. Columns of a matrix X are independent if for any v 0, Xv 0

42 Matrix rank Course outline The rank of a matrix is the maximum number of independent columns or rows in the matrix. Columns of a matrix X are independent if for any v 0, Xv 0 r(a) = r(a ) = r(a A) = r(aa ) r(ab) = min(r(a),r(b))

43 Matrix rank: example A = For matrix A, the three columns are independent and r(a) = 3. There is no v 0 such that Av = 0 (of course, if v = 0, Av = A0 = 0).

44 Matrix rank: example B = For matrix B the first and the last column vectors are linear combinations of each other: b 1 = 1 3 b 3. At most two columns (b 1 and b 2 or b 2 and b 3 ) are independent, so r(b) = 2. We could construct a matrix v such that Bv = 0, namely v = [ ] (or any v = [ α α] ).

45 Matrix rank: example C = r(c) = 2. In this case, one cannot express one of the column vectors as a linear combination of another column vector, but one can express any of the column vectors as a linear combination of the two other column vectors. For example, c 3 = 3c c 2 and thus when v = [ ], Cv = 0.

46 Matrix inverse Course outline The inverse of a matrix is the matrix one would have to multiply with to get the identity matrix, i.e.: A 1 A = AA 1 = I

47 Matrix inverse Course outline The inverse of a matrix is the matrix one would have to multiply with to get the identity matrix, i.e.: A 1 A = AA 1 = I (A 1 ) = (A ) 1 (AB) 1 = B 1 A 1

48 Matrices: trace The trace of a matrix is the sum of the diagonal elements A = tr(a) = sum(diag(a)) = = 16

49 Outline Course outline 1 Course outline 2 3

50 Expected value The expected value of a discrete random variable x is: E(x) = N P(x i ) x i, i whereby N is the total number of possible outcomes in S.

51 Expected value The expected value of a discrete random variable x is: E(x) = N P(x i ) x i, i whereby N is the total number of possible outcomes in S. The expected value is the mean (µ) of a random variable (not to be confused with the mean of a particular set of observations).

52 Moments Course outline moments are E(x) n, n = 1,2,... So µ x is the first moment of x central moments are E(x E(x)) n, n = 1,2,... absolute moments are E( x ) n, n = 1,2,... absolute central moments are E( x E(x) ) n, n = 1,2,...

53 Variance Course outline The second central moment is called the variance, so: var(x) = E(x E(x)) 2 = E(x µ x ) 2 N = P(x i ) (x i E(x)) 2 i

54 Variance Course outline The second central moment is called the variance, so: var(x) = E(x E(x)) 2 = E(x µ x ) 2 N = P(x i ) (x i E(x)) 2 i The standard deviation is the square root of the variance: sd(x) = var(x) = E(x E(x)) 2

55 Covariance Course outline The covariance of two random variables x and y is defined as: cov(x,y) = E[(x E(x))(y E(y))]

56 Covariance Course outline The covariance of two random variables x and y is defined as: cov(x,y) = E[(x E(x))(y E(y))] If x and y are independent of each other, cov(x,y) = 0.

57 Variance of a sum var(x +y) = var(x)+var(y)+2cov(x,y) var(x y) = var(x)+var(y) 2cov(x,y)

58 Exercise Course outline Let x be a random variable containing only ones and zeros. Let π be the probability of observing a one. Show that 1 E(x) = π 2 var(x) = π(1 π)