Chapter 4. Properties of the Least Squares Estimators. Assumptions of the Simple Linear Regression Model. SR3. var(e t ) = σ 2 = var(y t )

Transcription

1 Chaper 4 Properies of he Leas Squares Esimaors Assumpions of he Simple Linear Regression Model SR1. SR. y = β 1 + β x + e E(e ) = 0 E[y ] = β 1 + β x SR3. var(e ) = σ = var(y ) SR4. cov(e i, e j ) = cov(y i, y j ) = 0 SR5. SR6. x is no random and akes a leas wo values e ~ N(0, σ ) y ~ N[(β 1 + β x ), σ ] (opional) Undergraduae Economerics, nd Ediion Chaper 4 1

2 4.1 The Leas Squares Esimaors as Random Variables To repea an imporan passage from Chaper 3, when he formulas for b 1 and b, given in Equaion (3.3.8), are aken o be rules ha are used whaever he sample daa urn ou o be, hen b 1 and b are random variables since heir values depend on he random variable y whose values are no known unil he sample is colleced. In his conex we call b 1 and b he leas squares esimaors. When acual sample values, numbers, are subsiued ino he formulas, we obain numbers ha are values of random variables. In his conex we call b 1 and b he leas squares esimaes. Undergraduae Economerics, nd Ediion Chaper 4

3 4. The Sampling Properies of he Leas Squares Esimaors The means (expeced values) and variances of random variables provide informaion abou he locaion and spread of heir probabiliy disribuions (see Chaper.3). As such, he means and variances of b 1 and b provide informaion abou he range of values ha b 1 and b are likely o ake. Knowing his range is imporan, because our objecive is o obain esimaes ha are close o he rue parameer values. Since b 1 and b are random variables, hey may have covariance, and his we will deermine as well. These predaa characerisics of b 1 and b are called sampling properies, because he randomness of he esimaors is brough on by sampling from a populaion. Undergraduae Economerics, nd Ediion Chaper 4 3

4 4..1 The Expeced Values of b 1 and b The leas squares esimaor b of he slope parameer β, based on a sample of T observaions, is b ( ) T x y x y = T x x (3.3.8a) The leas squares esimaor b 1 of he inercep parameer β 1 is b1 = y bx (3.3.8b) are he sample means of he observaions on y where y = y / T and x = x / T and x, respecively. Undergraduae Economerics, nd Ediion Chaper 4 4

5 We begin by rewriing he formula in Equaion (3.3.8a) ino he following one ha is more convenien for heoreical purposes: b = β + we (4..1) where w is a consan (non-random) given by w = x x ( x x) (4..) Since w is a consan, depending only on he values of x, we can find he expeced value of b using he fac ha he expeced value of a sum is he sum of he expeced values (see Chaper.5.1): Undergraduae Economerics, nd Ediion Chaper 4 5

6 ( ) Eb ( ) = Eβ + we = E( β ) + Ewe ( ) = β + we( e) =β [since E( e) = 0] (4..3) When he expeced value of any esimaor of a parameer equals he rue parameer value, hen ha esimaor is unbiased. Since E(b ) = β, he leas squares esimaor b is an unbiased esimaor of β. If many samples of size T are colleced, and he formula (3.3.8a) for b is used o esimae β, hen he average value of he esimaes b obained from all hose samples will be β, if he saisical model assumpions are correc. However, if he assumpions we have made are no correc, hen he leas squares esimaor may no be unbiased. In Equaion (4..3) noe in paricular he role of he assumpions SR1 and SR. The assumpion ha E(e ) = 0, for each and every, makes Undergraduae Economerics, nd Ediion Chaper 4 6

7 we ( e ) = 0 and E(b ) = β. If E(e ) 0, hen E(b ) β. Recall ha e conains, among oher hings, facors affecing y ha are omied from he economic model. If we have omied anyhing ha is imporan, hen we would expec ha E(e ) 0 and E(b ) β. Thus, having an economeric model ha is correcly specified, in he sense ha i includes all relevan explanaory variables, is a mus in order for he leas squares esimaors o be unbiased. The unbiasedness of he esimaor b is an imporan sampling propery. When sampling repeaedly from a populaion, he leas squares esimaor is correc, on average, and his is one desirable propery of an esimaor. This saisical propery by iself does no mean ha b is a good esimaor of β, bu i is par of he sory. The unbiasedness propery depends on having many samples of daa from he same populaion. The fac ha b is unbiased does no imply anyhing abou wha migh happen in jus one sample. An individual esimae (number) b may be near o, or far from β. Since β is never known, we will never know, given one sample, wheher our 7 Undergraduae Economerics, nd Ediion Chaper 4

8 esimae is close o β or no. The leas squares esimaor b 1 of β 1 is also an unbiased esimaor, and E(b 1 ) = β a The Repeaed Sampling Conex To illusrae unbiased esimaion in a slighly differen way, we presen in Table 4.1 leas squares esimaes of he food expendiure model from 10 random samples of size T = 40 from he same populaion. Noe he variabiliy of he leas squares parameer esimaes from sample o sample. This sampling variaion is due o he simple fac ha we obained 40 differen households in each sample, and heir weekly food expendiure varies randomly. Undergraduae Economerics, nd Ediion Chaper 4 8

9 Table 4.1 Leas Squares Esimaes from 10 Random Samples of size T=40 n b 1 b The propery of unbiasedness is abou he average values of b 1 and b if many samples of he same size are drawn from he same populaion. The average value of b 1 in hese 10 samples is b 1 = The average value of b is b = If we ook he averages of esimaes from many samples, hese averages would approach he rue Undergraduae Economerics, nd Ediion Chaper 4 9

10 parameer values β 1 and β. Unbiasedness does no say ha an esimae from any one sample is close o he rue parameer value, and hus we can no say ha an esimae is unbiased. We can say ha he leas squares esimaion procedure (or he leas squares esimaor) is unbiased. 4..1b Derivaion of Equaion 4..1 In his secion we show ha Equaion (4..1) is correc. The firs sep in he conversion of he formula for b ino Equaion (4..1) is o use some ricks involving summaion signs. The firs useful fac is ha 1 ( x x) = x x x + T x = x x T x + T x T = x T x + T x = x T x (4..4a) Undergraduae Economerics, nd Ediion Chaper 4 10

11 Then, saring from Equaion(4..4a), ( x ) ( x x) = x T x = x x x = x (4..4b) T To obain his resul we have used he fac ha x = x / T, so x = T x. The second useful fac is x y ( x x)( y y) xy Tx y xy (4..5) T = = This resul is proven in a similar manner by using Equaion (4..4b). Undergraduae Economerics, nd Ediion Chaper 4 11

12 If he numeraor and denominaor of b in Equaion (3.3.8a) are divided by T, hen using Equaions (4..4) and (4..5) we can rewrie b in deviaion from he mean form as b = ( x x)( y y) ( x x) (4..6) This formula for b is one ha you should remember, as we will use i ime and ime again in he nex few chapers. Is primary advanage is is heoreical usefulness. The sum of any variable abou is average is zero, ha is, ( x x) = 0 (4..7) Then, he formula for b becomes Undergraduae Economerics, nd Ediion Chaper 4 1

13 b ( x x)( y y) ( x x) y y ( x x) = = ( x x) ( x x) ( x x) y ( x x) = = y = ( x x) ( x x) w y (4..8) where w is he consan given in Equaion (4..). To obain Equaion (4..1), replace y by y = β 1 + β x + e and simplify: (4..9a) b = w y = w( β +β x + e ) =β w +β wx + we 1 1 Undergraduae Economerics, nd Ediion Chaper 4 13

14 Firs, w = 0, his eliminaes he erm β 1 w. Secondly, wx = 1 (by using Equaion (4..4b)), so β wx =β, and (4..9a) simplifies o Equaion (4..1), which is wha we waned o show. b = β + we (4..9b) The erm w = 0, because ( x x) 1 w = ( ) 0 = x x = ( using ( x x) = 0) ( x x) ( x x) To show ha wx = 1 we again use ( x x) = 0 ( x x) is Undergraduae Economerics, nd Ediion Chaper 4. Anoher expression for 14

15 ( x x) = ( x x)( x x) = ( x x) x x ( x x) = ( x x) x Consequenly, wx ( x x) x ( x x) x = = = 1 ( x x) ( x x) x 4.. The Variances and Covariance of b 1 and b The variance of he random variable b is he average of he squared disances beween he values of he random variable and is mean, which we now know is E(b ) = β. The variance (Chaper.3.4) of b is defined as Undergraduae Economerics, nd Ediion Chaper 4 15

16 var(b ) = E[b E(b )] I measures he spread of he probabiliy disribuion of b. In Figure 4.1 are graphs of wo possible probabiliy disribuion of b, f 1 (b ) and f (b ), ha have he same mean value bu differen variances. The probabiliy densiy funcion f (b ) has a smaller variance han he probabiliy densiy funcion f 1 (b ). Given a choice, we are ineresed in esimaor precision and would prefer ha b have he probabiliy disribuion f (b ) raher han f 1 (b ). Wih he disribuion f (b ) he probabiliy is more concenraed around he rue parameer value β, giving, relaive o f 1 (b ), a higher probabiliy of geing an esimae ha is close o β. Remember, geing an esimae close o β is our objecive. The variance of an esimaor measures he precision of he esimaor in he sense ha i ells us how much he esimaes produced by ha esimaor can vary from sample o sample as illusraed in Table 4.1. Consequenly, we ofen refer o he sampling Undergraduae Economerics, nd Ediion Chaper 4 16

17 variance or sampling precision of an esimaor. The lower he variance of an esimaor, he greaer he sampling precision of ha esimaor. One esimaor is more precise han anoher esimaor if is sampling variance is less han ha of he oher esimaor. If he regression model assumpions SR1-SR5 are correc (SR6 is no required), hen he variances and covariance of b 1 and b are: Undergraduae Economerics, nd Ediion Chaper 4 17

18 x var( b1 ) =σ T( x x) var( b ) = σ ( x x) (4..10) x cov( b1, b) =σ ( x x) Le us consider he facors ha affec he variances and covariance in Equaion (4..10). 1. The variance of he random error erm, σ, appears in each of he expressions. I reflecs he dispersion of he values y abou heir mean E(y). The greaer he variance σ, he greaer is he dispersion, and he greaer he uncerainy abou Undergraduae Economerics, nd Ediion Chaper 4 18

19 where he values of y fall relaive o heir mean E(y). The informaion we have abou β 1 and β is less precise he larger is σ. The larger he variance erm σ, he greaer he uncerainy here is in he saisical model, and he larger he variances and covariance of he leas squares esimaors.. The sum of squares of he values of x abou heir sample mean, ( x x), appears in each of he variances and in he covariance. This expression measures how spread ou abou heir mean are he sample values of he independen or explanaory variable x. The more hey are spread ou, he larger he sum of squares. The less hey are spread ou he smaller he sum of squares. The larger he sum of squares, ( x x), he smaller he variance of leas squares esimaors and he more precisely we can esimae he unknown parameers. The inuiion behind his is demonsraed in Figure 4.. On he righ, in panel (b), is a daa scaer in which he values of x are widely spread ou along he x-axis. In panel (a) he daa are Undergraduae Economerics, nd Ediion Chaper 4 19

20 bunched. The daa in panel (b) do a beer job of deermining where he leas squares line mus fall, because hey are more spread ou along he x-axis. 3. The larger he sample size T he smaller he variances and covariance of he leas squares esimaors; i is beer o have more sample daa han less. The sample size T appears in each of he variances and covariance because each of he sums consiss of T erms. Also, T appears explicily in var(b 1 ). The sum of squares erm ( x x) ges larger and larger as T increases because each of he erms in he sum is posiive or zero (being zero if x happens o equal is sample mean value for an observaion). Consequenly, as T ges larger, boh var(b ) and cov(b 1, b ) ge smaller, since he sum of squares appears in heir denominaor. The sums in he numeraor and denominaor of var(b 1 ) boh ge larger as T ges larger and offse one anoher, leaving he T in he denominaor as he dominan erm, ensuring ha var(b 1 ) also ges smaller as T ges larger. Undergraduae Economerics, nd Ediion Chaper 4 0

21 4. The erm Σx appears in var(b 1 ). The larger his erm is, he larger he variance of he leas squares esimaor b 1. Why is his so? Recall ha he inercep parameer β 1 is he expeced value of y, given ha x = 0. The farher our daa from x = 0 he more difficul i is o inerpre β 1, and he more difficul i is o accuraely esimae β 1. The erm Σx measures he disance of he daa from he origin, x = 0. If he values of x are near zero, hen Σx will be small and his will reduce var(b 1 ). Bu if he values of x are large in magniude, eiher posiive or negaive, he erm Σx will be large and var(b 1 ) will be larger. 5. The sample mean of he x-values appears in cov(b 1, b ). The covariance increases he larger in magniude is he sample mean x, and he covariance has he sign ha is opposie ha of x. The reasoning here can be seen from Figure 4.. In panel (b) he leas squares fied line mus pass hrough he poin of he means. Given a fied line hrough he daa, imagine he effec of increasing he esimaed slope, b. Since he line mus pass hrough he poin of he means, he effec mus be o lower he 1 Undergraduae Economerics, nd Ediion Chaper 4

22 poin where he line his he verical axis, implying a reduced inercep esimae b 1. Thus, when he sample mean is posiive, as shown in Figure 4., here is a negaive covariance beween he leas squares esimaors of he slope and inercep. Deriving he variance of b : The saring poin is Equaion (4..1). var( b ) = var β + we = var we [since β is a consan] = w var( e) [using cov( ei, ej) = 0] = σ w [using var( e) =σ ] = ( ) ( ) σ ( x x) (4..11) Undergraduae Economerics, nd Ediion Chaper 4

23 The very las sep uses he fac ha w = = { ( x x) } ( x x) 1 ( x x) (4..1) Deriving he variance of b 1 : From Equaion (3.3.8b) b= y bx= 1 (β β x ) 1 1 e bx T + + = β + β x+ e b x= β ( b β ) x+ e 1 1 b β ( b β ) x+ e = 1 1 Undergraduae Economerics, nd Ediion Chaper 4 3

24 Since E(b ) = β and E() e = 0, i follows ha Eb ( ) = β (β β ) x+ 0= β Then var( b) = E[( b β )] = E[(( b β ) x+ e) ] = xe[( b β )] + Ee [ ] xe[( b β )] e = x var( b ) + E[ e ] xe[( b β )] e Now Undergraduae Economerics, nd Ediion Chaper 4 4

25 var( b ) = σ ( x ) x and Ee ( ) = E[( 1 e) ] 1 [( ) E e ] T = T = 1 E e + ee T [ cross-produc erms in i j] 1 { E [ e ] E [cross-produc erms in ee i j ]} = + T = 1 ( Tσ ) = 1 σ T T = = (Noe: var( e) [( ( )) ] [ E e E e E e ]) Undergraduae Economerics, nd Ediion Chaper 4 5

26 Finally, E[( b β ) e] = E[( )( 1 )] (from Equaion (4..9b)) we e T = E 1 we + ee T = 1 T = 1 σ T [ ( cross-produc erms in i j)] ( ) we e w = 0 ( w = 0) Therefore, Undergraduae Economerics, nd Ediion Chaper 4 6

27 var( b) = x E[( b β )] + E[ e ] 1 = σ x + σ ( x x) T = = = Tx + ( x x) σ [ ] T ( x ) x Tx + x Tx σ [ ] T ( x ) x x σ [ ] T ( x ) x Undergraduae Economerics, nd Ediion Chaper 4 7

28 Deriving he covariance of b 1 and b : cov( b, b ) = E[( b β )( b β )] = E[( e ( b β ))( x b β )] = Eeb xe b [ ( β )] [( β )] = σ x (Noe: Eb ( β ) = 0) ( x x) 4..3 Linear Esimaors The leas squares esimaor b is a weighed sum of he observaions y, b = w y [see Equaion (4..8)]. In mahemaics weighed sums like his are called linear Undergraduae Economerics, nd Ediion Chaper 4 8

29 combinaions of he y ; consequenly, saisicians call esimaors like b, ha are linear combinaions of an observable random variable, linear esimaors. Puing ogeher wha we know so far, we can describe b as a linear, unbiased esimaor of β, wih a variance given in Equaion (4..10). Similarly, b 1 can be described as a linear, unbiased esimaor of β 1, wih a variance given in Equaion (4..10). Undergraduae Economerics, nd Ediion Chaper 4 9

30 4.3 The Gauss-Markov Theorem Gauss-Markov Theorem: Under he assumpions SR1-SR5 of he linear regression model he esimaors b 1 and b have he smalles variance of all linear and unbiased esimaors of β 1 and β. They are he Bes Linear Unbiased Esimaors (BLUE) of β 1 and β. Le us clarify wha he Gauss-Markov heorem does, and does no, say. 1. The esimaors b 1 and b are bes when compared o similar esimaors, hose ha are linear and unbiased. The Theorem does no say ha b 1 and b are he bes of all possible esimaors.. The esimaors b 1 and b are bes wihin heir class because hey have he minimum variance. Undergraduae Economerics, nd Ediion Chaper 4 30

31 3. In order for he Gauss-Markov Theorem o hold, he assumpions (SR1-SR5) mus be rue. If any of he assumpions 1-5 are no rue, hen b 1 and b are no he bes linear unbiased esimaors of β 1 and β. 4. The Gauss-Markov Theorem does no depend on he assumpion of normaliy (assumpion SR6). 5. In he simple linear regression model, if we wan o use a linear and unbiased esimaor, hen we have o do no more searching. The esimaors b 1 and b are he ones o use. 6. The Gauss-Markov heorem applies o he leas squares esimaors. I does no apply o he leas squares esimaes from a single sample. Proof of he Gauss-Markov Theorem: Le b = k y (where he k are consans) be any oher linear esimaor of β. * Suppose ha k = w + c, where c is anoher consan and w is given in Equaion Undergraduae Economerics, nd Ediion Chaper 4 31

32 (4..). While his is ricky, i is legal, since for any k ha someone migh choose we can find c. Ino his new esimaor subsiue y and simplify, using he properies of w, w = 0 and w x = 1. Then i can be shown ha b = k y = ( w + c ) y = ( w + c )( β +β x + e ) * 1 = ( w + c ) β + ( w + c ) β x + ( w + c ) e 1 =β w +β c +β wx +β c x + ( w + c ) e 1 1 (4.3.1) =β c +β +β c x + ( w + c ) e 1 Undergraduae Economerics, nd Ediion Chaper 4 3

33 Take he mahemaical expecaion of he las line in Equaion (4.3.1), using he properies of expecaion (see Chaper.5.1) and he assumpion ha E(e ) = 0. Eb ( ) =β c+β +β cx+ ( w+ c) Ee ( ) * 1 =β c +β +β c x 1 (4.3.) In order for he linear esimaor b = k y o be unbiased, i mus be rue ha * c = 0 and cx = 0 (4.3.3) Undergraduae Economerics, nd Ediion Chaper 4 33

34 These condiions mus hold in order for b * = ky o be in he class of linear and unbiased esimaors. So we will assume he condiions (4.3.3) hold and use hem o simplify expression (4.3.1): (4.3.4) * b = ky =β + ( w + c) e We can now find he variance of he linear unbiased esimaor b * following he seps in Equaion (4..11) and using he addiional fac ha c( x x) 1 x cw = 0 = cx c = ( x x) ( x x) ( x x) (The las sep is from condiions (4.3.3)) Undergraduae Economerics, nd Ediion Chaper 4 34

35 Use he properies of variance o obain: * ( ) var( b ) = var β + ( w + c ) e = ( w + c ) var( e ) w c w c cw =σ ( + ) =σ +σ (Noe: σ = 0) = var( b ) +σ c (4.3.5) var( b ) since c 0 (and 0) σ > The las line of Equaion (4.3.5) esablishes ha, for he family of linear and unbiased esimaors * b, each of he alernaive esimaors has variance ha is greaer han or equal o ha of he leas squares esimaor b. The only ime ha var( b* ) = var( b ) is Undergraduae Economerics, nd Ediion Chaper 4 35

36 when all he c = 0, in which case b * = b. Thus, here is no oher linear and unbiased esimaor of β ha is beer han b, which proves he Gauss-Markov heorem. Undergraduae Economerics, nd Ediion Chaper 4 36

37 4.4 The Probabiliy Disribuion of he Leas Squares Esimaors If we make he normaliy assumpion, ha he random errors e are normally disribued wih mean 0 and variance σ, hen he probabiliy disribuions of he leas squares esimaors are also normal. b b 1 ~ N β1, T ( x x) ~ N β, σ σ ( x x) x (4.4.1) This conclusion is obained in wo seps. Firs, based on assumpion SR1, if e is normal, hen so is y. Second, he leas squares esimaors are linear esimaor, of he 37 Undergraduae Economerics, nd Ediion Chaper 4

38 erm b = w y, and weighed sums of normal random variables, using Equaion (.6.4), are normally disribued hemselves. Consequenly, if we make he normaliy assumpion, assumpion SR6 abou he error erm, hen he leas squares esimaors are normally disribued. If assumpions SR1-SR5 hold, and if he sample size T is sufficienly large, hen he leas squares esimaors have a disribuion ha approximaes he normal disribuions shown in Equaion (4.4.1). Undergraduae Economerics, nd Ediion Chaper 4 38

39 4.5 Esimaing he Variance of he Error Term The variance of he random variable e is he one unknown parameer of he simple linear regression model ha remains o be esimaed. The variance of he random variable e (see Chaper.3.4) is var( e) E[( e E( e)) ] E( e ) =σ = = (4.5.1) if he assumpion E(e ) = 0 is correc. Since he expecaion is an average value we migh consider esimaing σ as he average of he squared errors, e ˆ σ = (4.5.) T Undergraduae Economerics, nd Ediion Chaper 4 39

40 The formula in Equaion (4.5.) is unforunaely of no use, since he random error e are unobservable! While he random errors hemselves are unknown, we do have an analogue o hem, namely, he leas squares residuals. Recall ha he random errors are e = y β 1 β x and he leas squares residuals are obained by replacing he unknown parameers by heir leas squares esimaors, e = y b b x ˆ 1 I seems reasonable o replace he random errors e in Equaion (4.5.) by heir analogues, he leas squares residuals, o obain Undergraduae Economerics, nd Ediion Chaper 4 40

41 ˆ e ˆ σ = (4.5.3) T Unforunaely, he esimaor in Equaion (4.5.3) is a biased esimaor of σ. Happily, here is a simple modificaion ha produces an unbiased esimaor, and ha is ˆ ˆ e σ = (4.5.4) T The ha is subraced in he denominaor is he number of regression parameers (β 1, β ) in he model. The reason ha he sum of he squared residuals is divided by T is ha while here are T daa poins or observaions, he esimaion of he inercep and slope pus wo consrains on he daa. This leaves T unconsrained Undergraduae Economerics, nd Ediion Chaper 4 41

42 observaions wih which o esimae he residual variance. This subracion makes he esimaor ˆσ unbiased, so ha ˆ E( σ ) =σ (4.5.5) Consequenly, before he daa are obained, we have an unbiased esimaion procedure for he variance of he error erm, σ, a our disposal Esimaing he Variances and Covariances of he Leas Squares Esimaors Replace he unknown error variance σ in Equaion (4..10) by is esimaor o obain: Undergraduae Economerics, nd Ediion Chaper 4 4

43 x var(? b ˆ 1) =σ, se( b1) = var( b1) T ( x x) σˆ var(? b ) =, se( b ) = var( b ) ( x x) x cov( ˆ b ˆ 1, b) =σ ( x x) (4.5.6) Therefore, having an unbiased esimaor of he error variance, we can esimae he variances of he leas squares esimaors b 1 and b, and he covariance beween hem. The square roos of he esimaed variances, se(b 1 ) and se(b ), are he sandard errors of b 1 and b. Undergraduae Economerics, nd Ediion Chaper 4 43

44 4.5. The Esimaed Variances and Covariances for he Food Expendiure Example The leas squares esimaes of he parameers in he food expendiure model are given in Chaper In order o esimae he variance and covariance of he leas squares esimaors, we mus compue he leas squares residuals and calculae he esimae of he error variance in Equaion (4.6.4). In Table 4. are he leas squares residuals for he firs five households in Table 3.1. Table 4. Leas Squares Residuals for Food Expendiure Daa ŷ= b + b x?e = y y y Undergraduae Economerics, nd Ediion Chaper 4 44

45 Using he residuals for all T = 40 observaions, we esimae he error variance o be e ˆ σ ˆ = = = T 38 The esimaed variances, covariances and corresponding sandard errors are x var( ˆ b ˆ 1) =σ = = T ( x x) 40(153463) se( b) = var( ˆ b) = = x x ˆ σ var( ˆ b ) = = = ( ) Undergraduae Economerics, nd Ediion Chaper 4 45

46 se( b ) = var( ˆ b ) = = x 698 cov( ˆ b ˆ 1, b) =σ = = ( x x) Undergraduae Economerics, nd Ediion Chaper 4 46

47 4.5.3 Sample Compuer Oupu Dependen Variable: FOODEXP Mehod: Leas Squares Sample: 1 40 Included observaions: 40 Variable Coefficien Sd. Error -Saisic Prob. C INCOME R-squared Mean dependen var Adjused R-squared S.D. dependen var S.E. of regression Akaike info crierion Sum squared resid Schwarz crierion Log likelihood F-saisic Durbin-Wason sa Prob(F-saisic) Table 4.3 EViews Regression Oupu Undergraduae Economerics, nd Ediion Chaper 4 47

48 Dependen Variable: FOODEXP Analysis of Variance Sum of Mean Source DF Squares Square F Value Prob>F Model Error C Toal Roo MSE R-square Dep Mean Adj R-sq C.V Parameer Esimaes Parameer Sandard T for H0: Variable DF Esimae Error Parameer=0 Prob > T INTERCEP INCOME Table 4.4 SAS Regression Oupu Undergraduae Economerics, nd Ediion Chaper 4 48

49 VARIANCE OF THE ESTIMATE-SIGMA** = 149. VARIABLE ESTIMATED STANDARD NAME COEFFICIENT ERROR X E-01 CONSTANT Table 4.5 SHAZAM Regression Oupu Covariance of Esimaes COVB INTERCEP X INTERCEP X Table 4.6 SAS Esimaed Covariance Array Appendix Undergraduae Economerics, nd Ediion Chaper 4 49

50 Deriving Equaion (4.5.4) eˆ = y b b x = (β + β x + e) b b x = ( b β ) ( b β ) x + e 1 1 = ( b β ) x e ( b β ) x + e (see derivaion of var( b)) 1 = ( b β )( x x) + ( e e) eˆ = ( b β )( x x) + ( e e) ( b β )( x x)( e e) = ( b β ) ( x x) + ( e e) ( b β )( x x) e + ( b β )( x x) e eˆ = ( b β ) ( x x) + ( e e) ( b β ) ( x xe ) + ( b β ) ( x xe ) = ( b β ) ( x x) + ( e e) ( b β ) ( x x) e ( Q ( x x) e = 0 ) Undergraduae Economerics, nd Ediion Chaper 4 50

51 Noe: ( x x) e b β (from Equaions (4..1) and (4..) = we = ( x ) x ( x x) e = ( b β ) ( x x) Noe: ( e e) = ( e ee+ e ) = e e e + Te = e ( 1 e)( e) T( 1 + e) T T = ( ) 1 ( ) 1 e e e e ( e) T + = T T = e 1 ( e + ee ) = T 1 e ee T i j i j i j T T i j Undergraduae Economerics, nd Ediion Chaper 4 51

52 Now, eˆ = ( b β ) ( x x) + ( e e) = ( b β ) ( x x) + T 1 e ee T T i j i i E[ eˆ ] = E[( b β )] ( x x) + T 1 E[ e ] E( ee ) T T i j i i = E b x x T 1 + E e E ee = T Q [( β )] ( ) [ ] ( ( ) 0) i j = T 1 + T var( b ) ( x ) ( σ ) x T = σ + ( ) ( x ) ( 1)σ (from Equaion (4..11)) ( ) x T x x = σ + ( T 1)σ = ( T )σ Undergraduae Economerics, nd Ediion Chaper 4 5

53 Finally, E? = 1 E e T [σ ] [ ] = 1 ( T )σ = σ T Undergraduae Economerics, nd Ediion Chaper 4 53

54 Exercise Undergraduae Economerics, nd Ediion Chaper 4 54