Lecture 2: The Classical Linear Regression Model

Transcription

1 Lecture : The Classcal Lnear Regresson Model Introducton In lecture we Introduced concept of an economc relaton Noted that emprcal economc relatons do not resemble tetbook relatons Introduced a method to fnd the best fttng lnear relaton No reference to mathematcal statstcs n all of ths. Intally econometrcs dd not use the tools of mathematcal statstcs. Mathematcal statstcs develops methods for the analyss of data generated by a random eperment n order to learn about that random eperment.

2 Is ths relevant n economcs? Consder Wage equaton: relaton between wage and educaton, work eperence, gender, Macro consumpton functon: relaton between natonal consumpton and natonal ncome What s the random eperment? To make progress we start wth the assumpton that all economc relatons are essentally determnstc, f we nclude all varables y f,, W

3 Hence, f we have data y,,,,, n then W, y f,, W,,, n Let,, W be the sample averages of the varables and assume that f s suffcently many tmes dfferentable to have a Taylor seres epanson around,, W,.e. a polynomal appromaton y W W W W W + γ δ + 3

4 We dvde,, W nto three groups. Varables that do not vary n the sample take ths to be last W V varables,.e. for,, n,, V + V +,,, W W. Eample: gender f we consder a sample of women.. Varables n the relaton that are omtted or cannot be ncluded because they are unobservable. Let ths be the net V K + varables. 3. Varables ncluded n the relaton,.e.,, K +. 4

5 To keep the relaton smple we concentrate on the lnear part. Hence the observatons satsfy y K, K + ε wth K 0 0 j j j The remander term contans all the omtted terms ε K K K V V V + γ + + γ V V V + + δ + We call ε the dsturbance of the eact lnear relaton. 5

6 Note that j f,, W, j,, K j Conclusons: K j f,, W 0 j j. The slope coeffcent j s the partal effect of j on y.. The slope coeffcents and the ntercept depend on the varables that are constant n a sample, e.g. n a sample of women the coeffcents n a wage relaton may be dfferent from those n a sample of only men. 3. Only n very specal cases wll the relaton have a 0 ntercept. 6

7 Consder the followng eperment: Predcton of y on the bass of.,,, K,, Assume that we have observed, K. Ths does not tell us anythng about the dsturbance ε that depends on many varables besde,, K. Hence, even f we know the coeffcents 0,, K, we cannot predct wth certanty what y s. A varable wth a value that s unknown before the eperment s performed s a random varable n probablty theory. As n classcal random eperments flppng con, rollng de randomness due to lack of knowledge. 7

8 In predcton eperment ys a random varable and hence so s ε y 0 K, K If we have a sample of sze n, we have n replcatons of the random eperment y y n n K K, K n, K + ε + ε n The dstrbuton of the dsturbance ε reflects the fact that we do not know the value of the dsturbance f we know,, K. In fact we make a more etreme assumpton: the value of ε s completely unpredctable from,, K. 8

9 Assumpton : E ε,,, K 0 In words: the dsturbance ε s mean-ndependent of,, K. Why do we need ths assumpton? Consder ε γ + η,, n, Hence, ε s partally predctable usng but not completely so. Substtuton gves y 0 γ + + γ + + K, K + η 9

10 Remember that f,, W Concluson: If holds then the coeffcent of s not equal to the partal effect of on y. Falure of Assumpton s a falure of the ceters parbus condton n the sample: a change n has two effects on y, a drect effect and an ndrect effect γ. The latter effect s due to the fact that n a sample we cannot hold other relevant varables fed/constant. Hence we only measure the partal effect f the omtted varables are not related wth. Measurng partal effects s the goal of most emprcal research n economcs and other socal scences. The bggest challenge n emprcal research s to ensure the Assumpton holds. 0

11 Whether we care depends on our objectves. Compare homeowner who s nterested n relaton between house prce and square footage of house. If he/she wants to predct the sales prce of house as s no reason to be worred about nterpretaton of regresson coeffcent as partal effect. If he/she wants to evaluate the nvestment n an addton to the house the estmaton of the partal effect s essental. Two strateges to estmate partal effect Include all varables that are correlated wth n the relaton. Assgn randomly,.e. usng a random eperment that s ndependent of anythng, e.g. by flppng a con f s dchotomous.

12 The CLR model Re-label the varables,, K as,, K and ntroduce the varable.e.,,, n. Also re-label 0,, K to,, K. In the new notaton for,, n y K K + ε Ths s the multple lnear regresson model. The relaton s lnear n,, K and also lnear n,, K. The latter s essental. By renterpretng,, K we can deal wth relatons that are non-lnear n these varables.

13 Eamples: Polynomal n 3 y ε 3 4 Defne:,, 3, 4 3 3

14 Log-lnear relaton ln y + ln + ε Defne:, ln Note ln y ln.e. s the elastcty of y w.r.t.. 4

15 Sem-log relaton ln y + + ε Note: no re-defnton s needed. Also y ln y y Ths s a sem-elastcty. 5

16 Assumpton n the new notaton: 3 E ε,, 0,, n K, Note that ths mples that E ε 0. Ths s wthout loss of generalty because a non-zero mean can be absorbed nto the ntercept. 6

17 In matr notaton and 3 are and y + ε wth E ε 0,,, n 4 n Note that s a K vector. 7

18 The Classcal Multple Lnear Regresson CLR model s a set of assumptons manly on the condtonal dstrbuton of ε gven,, K. Some assumptons are essental, some are convenent ntal assumptons and can/wll be relaed. The CLR model s approprate n many practcal stuatons and s the startng pont for the use of mathematcal statstcal nference to measure economc relatons 8

19 CLR model y + ε Assumpton : Fundamental assumpton E ε 0 Assumpton : Sphercal dsturbances E ε ε σ Assumpton 3: Full rank I rank K 9

20 Dscusson of the assumptons Assumpton s shorthand for E ε 0,,, n Hence ths s equvalent to E ε,, n 0,,, n Compare ths wth E ε 0,,, n By the law of terated epectatons, the current assumpton mples the latter. 0

21 The current assumpton states that not only but also j, j s not related to ε. Ths s not stronger than the prevous assumpton f,,n are ndependent as n a random sample from a populaton. If these are not ndependent, as e.g. n tme-seres data, then ths addtonal assumpton may be too strong. Assumpton s satsfed f s chosen ndependently of ε. In that case we can treat as a matr of known constants. Therefore nstead of Assumpton one sometmes sees Assumpton : s a matr of known constants determned ndependently of ε. Note: Chosng/controllng s not enough.

22 Net, we consder assumpton Note ' n n n ε ε ε ε ε ε ε ε ε ε εε Hence Assumpton mples that n,,, E σ ε Ths s called homoskedastcty.

23 Assumpton also mples that E ε ε j 0 Hence, gven the dsturbances are uncorrelated. 3

24 Eample of falure of homoskedastcty: random coeffcents y + + u + ε 0 populaton varaton n coeffcent + + ε + u 0 For the composte dsturbance, f E u 0 E ε + u 0 but Var ε + u σ + σ ε u + σ u wth σ ε E εu, σ Eu. Hence the composte error s heteroskedastc. u u 4

25 Falure of uncorrelated dsturbances: seral correlaton Assume seral correlaton of order n dsturbances ε ρε + u Apples n tme-seres. Fnally, consder Assumpton 3. If rank K, then a 0 f and only f a 0 wth a a K vector,.e. there s no lnear relaton between the K varables. 5

26 Falure of Assumpton 3: wage equaton y ε,,, n wth 3 4 schoolng years age potental eperence age-years n school-6 6

27 Hence

28 8 Defne for any constant, c c c c c ~, ~, ~, 6 ~ Then also n y,,, ~ ~ ~ ~ ε Concluson: We cannot dstngush between 4,, and 4 ~,, ~. For all c these parameters are observatonally equvalent. Problem s also clear f we substtute for 4 y ε

29 By assumptons and, E ε 0, Var ε σ I. Sometmes t s assumed that I ε ~ N0, σ. Why s the normal dstrbuton a natural choce? In sequel we sometmes assume: Assumpton 4 ε ~ N0, σ I 9

30 Lnear regresson as projecton Alternatve nterpretaton of lnear regresson s as condtonal mean functon. In prevous dervaton regresson coeffcent s the partal effect on y j n multple lnear regresson f j,, W j For ths result Assumpton was necessary. Alternatve: Consder j as the coeffcent of jn a lnear condtonal mean functon. To keep thngs smple we consder the case of eplanatory varable and an ntercept. 30

31 The dependent varable y and the ndependent varable have a jont populaton dstrbuton wth jont frequency dstrbuton f, y. Eample: y savngs rate, ncome and f, y s frequency dstrbuton over all US households. By a sample survey we obtan y,,,, n. Ths survey can be used to obtan an estmate of the populaton f, y denoted by fˆ, y see table.. Note savngs rate and ncome have been dscretzed. 3

32 3

33 From the jont frequency n the sample we obtan the sample condtonal frequency dstrbuton see table. fˆy, fˆ y fˆ 33

34 If there s an eact relaton between y and, then every column should have one and rest 0 s. If not, we can consder the average of y for every value of. Ths s the condtonal mean functon. In populaton wth estmate m E y y f y m ˆ y fˆ y y y 34

35 Note that m ˆ may be rough: samplng varaton around smooth populaton m. 35

36 Why s m E y nterestng? One reason s optmal predcton. Assume jont populaton dstrbuton f, y s known and that you have a random draw from ths dstrbuton. Only s revealed and you must predct y. What s the best predctor h? Crteron: mnmze epected squared predcton error E y h E y m + m h y m + E y m m h E + m h E y m + E and ths lower bound s acheved f h m. Concluson: Optmal predcton s h E y. 36

37 Now let us restrct to lnear predcton h a + b Best lnear predctor mne a, b y a b Frst-order condtons wth u y a b E u 0 E y a + b E E u 0 E y a + b E 37

38 38 Soluton compare wth OLS soluton Var, Cov y b E E b y a If we replace populaton moments wth sample moments we obtan n n y y b ˆ b y a ˆ ˆ Ths s the OLS soluton!

39 There s an mportant dfference between the case that the condtonal mean s lnear and the case that the condtonal mean s not lnear. If E y a + b, we have for u y a b y E y 0 E u E y a b E Ths s Assumpton n the CLR model. However, b has not a structural nterpretaton, as a partal effect! 39

40 If condtonal mean s not lnear, we have from the frst-order condton E u 0. Ths s weaker: but E u 0 E u 0 E u 0 not E u 0 Hence n that case Assumpton of the CLR model s not satsfed but a weaker uncorrelatedness assumpton. 40

41 The Ordnary Least Squares OLS estmator We have n observatons y,,, K,,, n. We organze the data n the n vector y and the n K matr wth n K nk The observatons are a sample from a populaton and we assume that the jont dstrbuton of y, s such that the CLR model s the approprate statstcal model,.e. y, satsfy y + ε 4

42 for some K vector of regresson coeffcents and some n vector of random errors ε wth a dstrbuton that satsfes E ε 0 E ε ε σ I Ths specfes the random eperment of whch CLR model. y, s the outcome the We can now dscuss statstcal nference: estmaton of populaton parameters and tests of hypotheses concernng the populaton parameters and other aspects of the populaton dstrbuton. 4

43 Setup apples to both cross-secton and tme-seres data. In frst case y,,, K,,, n s a random sample and the CLR assumptons on the populaton dstrbuton can made on y + ε Because the observatons are ndependent we can obtan the jont dstrbuton of y, from the margnal dstrbutons. For tme-seres data the observatons are not ndependent and the CLR model apples drectly to the jont dstrbuton of y,. 43

44 Estmaton of and σ The soluton to mnmzaton of sum of squared devatons/resduals b ' ' y Note that for ths rank K. Ths s an estmator of only depends on the data: the Ordnary Least Squares OLS estmator of. Is the OLS estmator a good estmator? In mathematcal statstcs estmators are evaluated by consderng ther samplng dstrbuton,.e. ther dstrbuton n repeated samples y,, s, S. s s, 44

45 All samples are realzatons of CLR random eperment y + ε, s, s s s, S The samplng dstrbuton of the OLS estmator b s the frequency dstrbuton of b s, s,, S for S large. We can obtan ths dstrbuton by computer smulaton as n assgnment. 45

46 46 Alternatve s to use the CLR assumptons and rules of probablty theory to derve features of the samplng dstrbuton of b. Consder ε ε ' ' ' ' ' ' y b + + From ths we can, usng the CLR assumptons, fnd the condtonal average of b gven n the samplng dstrbuton ε + ' ' E b E Hence the uncondtonal average of b s law of terated epectatons b E E b E In words: under CLR assumptons the OLS estmator s unbased for.

47 Besde mean consder the varance of b: Var b [ b E b b E b' ] E[ b '] E b We have b ' 'ε Upon substtuton ' ' ' ' Var b E εε 47

48 48 We have ' ' ' ' ' ' ' ' ' E E σ εε εε and hence ' ' Var E E b σ σ Note that ' σ s an unbased estmator of ths varance.

49 In specal case of constant and one regressor we have the unbased varance estmator Var b n σ Note that ths decreases wth σ and wth the varaton n. 49

50 Optmalty of OLS estmator n CLR model Consder class of estmators for that are lnear n y,.e. b L Cy For the OLS estmator C ' ',.e. C n general depends on. Gauss-Markov Theorem: In CLR model the OLS estmator s the Best Lnear Unbased BLU of,.e. t has the smallest varance of lnear unbased estmators. 50