3 Multple lear regresso: estmato ad propertes Ezequel Urel Uversdad de Valeca Verso: 09-013 3.1 The multple lear regresso model 1 3.1.1 Populato regresso model ad populato regresso fucto 3.1. Sample regresso fucto 4 3. Obtag the OLS estmates, terpretato of the coeffcets, ad other characterstcs 4 3..1 Obtag the OLS estmates 4 3.. Iterpretato of the coeffcets 6 3..3 Algebrac mplcatos of the estmato 10 3.3 Assumptos ad statstcal propertes of the OLS estmators 11 3.3.1 Statstcal assumptos of the CLM multple lear regresso) 11 3.3. Statstcal propertes of the OLS estmator 13 3.4 More o fuctoal forms 17 3.4.1 Use of logarthms the ecoometrc models 17 3.4. Polyomal fuctos 18 3.5 Goodess-of-ft ad selecto of regressors. 19 3.5.1 Coeffcet of determato 0 3.5. Adusted R-Squared 1 3.5.3 Akake formato crtero (AIC) ad Schwarz crtero (SC) 1 Exercses 3 Appedxes 3 Appedx 3.1 Proof of the theorem of Gauss-Markov 3 Appedx 3. Proof: s a ubased estmator of the varace of the dsturbace 33 Appedx 3.3 Cosstecy of the OLS estmator 34 Appedx 3.4 Maxmum lkelhood estmator 35 3.1 The multple lear regresso model The smple lear regresso model s ot adequate for modelg may ecoomc pheomea, because order to expla a ecoomc varable t s ecessary to take to accout more tha oe relevat factor. We wll llustrate ths wth some examples. I the Keyesa cosumpto fucto, dsposable come s the oly relevat varable: cos c u (3-1) 1 However, there are other factors that may be cosdered relevat cosumer behavor. Oe of these factors could be wealth. By cludg ths factor, we wll have a model wth two explaatory varables: cos c wealth u (3-) 1 3 I the aalyss of producto, a potetal fucto s ofte used, whch ca be trasformed to a lear model the parameters wth a adequate specfcato (takg atural logs). Usg a sgle put -labor- a model of ths type would be specfed as follows: l( output) l( labor) u (3-3) 1 The prevous model s clearly suffcet for ecoomc aalyss. It would be better to use the well-kow Cobb-Douglas model that cosders two puts (labor ad captal): 1
l( output) l( labor) l( captal) u (3-4) 1 3 Accordg to mcroecoomc theory, total costs (costot) are expressed as a fucto of the quatty produced (quatprod). A frst approxmato to expla the total costs could be a model wth oly oe regresor: costot quatprod u (3-5) 1 However, t s very restrctve cosderg that, as would be the case wth the prevous model, the margal cost remas costat regardless of the quatty produced. I ecoomc theory, a cubc fucto s proposed, whch leads to the followg ecoometrc model: costot quatprod quatprod quatprod u (3-6) 3 1 3 4 I ths case, ulke the prevous oes, oly oe explaatory varable s cosdered, but wth three regressors. Wages are determed by several factors. A relatvely smple model could expla wages usg years of educato ad years of experece as explaatory varables: wages educ exper u (3-7) 1 3 Other mportat factors to expla wages receved ca also be quattatve varables such as trag ad age, or qualtatve varables, such as sex, dustry, ad so o. Fally, explag the expedture o fsh relevat factors are the prce of fsh, the prce of a substtutve commodty such as meat, ad dsposable come: fshexp fshprce meatprce come u (3-8) 1 3 4 Thus, the above examples hghlght the eed for usg multple regresso models. The ecoometrc treatmet of the smple regresso model was made wth ordary algebra. The treatmet of a ecoometrc model wth two explaatory varables by usg ordary algebra s tedous ad cumbersome. Moreover, a model wth three explaatory varables s vrtually tractable wth ths tool. For ths reaso, the regresso model wll be preseted usg matrx algebra. 3.1.1 Populato regresso model ad populato regresso fucto I the model of multple lear regresso, the regressad (whch ca be ether the edogeous varable or a trasformato of the edogeous varables) s a lear fucto of k regressors correspodg to the explaatory varables -or ther trasformatos - ad of a radom dsturbace or error. The model also has a tercept. Desgatg the regressad by y, the regressors by x, x 3,..., x k ad the dsturbace or the radom dsturbace- by u, the populato model of multple lear regresso s gve by the followg expresso: y x x x u (3-9) 1 3 3 k k+ The parameters 1,, 3,, k are fxed ad ukow.
O the rght had of (3-9) we ca dstgush two parts: the systematc compoet 1x 3x3 kxk ad the radom dsturbace u. Callg y to the systematc compoet, we ca wrte: x x x (3-10) y 1 3 3 k k Ths equato s kow as the populato regresso fucto (PRF) or populato hyperplae. Whe k= the PRF s specfcally a straght le; whe k=3 the PRF s specfcally a plae; fally, whe k>3 the PRF s geercally deomated hyperplae. Ths caot to be represeted a three dmeso space. Accordg to (3-10), y s a lear fucto of the parameters 1,, 3,, k. Now, let us suppose we have a radom sample of sze {( y, x, x3,, xk) : 1,,, } extracted from the populato studed. If we wrte the populato model for all observatos of the sample, the followg system s obtaed: y1 1x13x31 kxk1u1 y x x x u y x x x u 1 3 3 k k 1 3 3 k k (3-11) The prevous system of equatos ca be expressed a compact form by usg matrx otato. Thus, we are gog to deote y1 y y... y 1 x1 x31... xk1 1 x x3... x k X 1 x x3... xk 1 3 k u1 u u... u The matrx X s called the matrx of regressors. Also cluded amog the regressors s the regressor correspodg to the tercept. Ths oe, whch s ofte called dummy regressor, takes the value 1 for all the observatos. The model of multple lear regresso (3-11) expressed matrx otato s the followg: 1 y1 1 x1 x31... xk1 u1 y 1 x x3... x k u 3 y 1 x x3... x k u k (3-1) If we take to accout the deomatos gve to vectors ad matrces, the model of multple lear regresso ca be expressed the followg way: y=x +u (3-13) where y s a vector 1, X s a matrx k, s a vector k 1 ad u s a vector 1. 3
3.1. Sample regresso fucto The basc dea of regresso s to estmate the populato parameters, 1,, 3,, k from a gve sample. The sample regresso fucto (SRF) s the sample couterpart of the populato regresso fucto (PRF). Sce the SRF s obtaed for a gve sample, a ew sample wll geerate dfferet estmates. The SRF, whch s a estmato of the PRF, s gve by yˆ ˆ ˆ x ˆ x ˆ x 1,,, (3-14) 1 3 3 k k The above expresso allows us to calculate the ftted value ( y ˆ ) for each y. I the SRF ˆ 1, ˆ ˆ ˆ, 3,, k are the estmators of the parameters 1,, 3,, k. We call resdual to the dfferece betwee y ad y. That s uˆ y yˆ y ˆ ˆ x ˆ x ˆ x (3-15) 1 3 3 k k I other words, the resdual u ˆ s the dfferece betwee a sample value ad ts correspodg ftted value. The system of equatos (3-14) ca be expressed a compact form by usg matrx otato. Thus, we are gog to deote ˆ 1 yˆ 1 ˆ uˆ 1 yˆ y ˆ ˆ ˆ uˆ... 3 u ˆ... yˆ uˆ ˆk For all observatos of the sample, the correspodg ftted model wll be the followg: ŷ =X ˆ (3-16) The resdual vector s equal to the dfferece betwee the vector of observed values ad the vector of ftted values, that s to say, uˆ y - y ˆ = y -X ˆ (3-17) 3. Obtag the OLS estmates, terpretato of the coeffcets, ad other characterstcs 3..1 Obtag the OLS estmates Deotg S to the sum of the squared resduals, ˆ ˆ ˆ ˆ ˆ 1 3 3 k k (3-18) 1 1 S u y x x x ˆ 4
to apply the least squares crtero the model of multple lear regresso, we calculate the frst dervatve from S wth respect to each ˆ the expresso (3-18): S ˆ ˆ ˆ ˆ 1 3 3 1 ˆ y x x kx k 1 1 S ˆ ˆ ˆ ˆ 1 3 3 k k ˆ y x x x x 1 S y ˆ ˆ ˆ ˆ 1 x 3x3 kxk x3 ˆ 3 1 S y ˆ ˆ ˆ ˆ 1 x 3x3 k xk ˆ xk k 1 The least square estmators are obtaed equalg to 0 the prevous dervatves: 1 1 y ˆ ˆ ˆ ˆ 1 x 3x3 kx k 0 y ˆ ˆ ˆ ˆ 1 x 3x3 kx k x 0 y ˆ ˆ ˆ ˆ 1 x 3x3 kx k x3 0 1 1 y ˆ ˆ 1x ˆ ˆ 3x3 kx k xk 0 (3-19) (3-0) or, matrx otato, XX ˆ Xy (3-1) The prevous equatos are deomated geercally hyperplae ormal equatos. I expaded matrx otato, the system of ormal equatos s the followg: x... xk y 1 1 ˆ 1 1 ˆ x x xxk x y 1 1 1 1 ˆ k xk xkx x k xk y 1 1 1 1 (3-) Note that: a) XX / s the matrx of secod order sample momets wth respect to the org, of the regressors, amog whch a dummy regressor (x 1 ) assocated to the tercept s cluded. Ths regressor takes the value x 1 =1 for all. 5
b) Xy / s the vector of sample momets of secod order, wth respect to the org, betwee the regressad ad the regressors. I ths system there are k equatos ad k ukow ( ˆ ˆ ˆ ˆ 1,, 3,, k ). Ths system ca easly be solved usg matrx algebra. I order to solve uvocally the system (3-1)wth respect to ˆ, t must be held that the rak of the matrx XX s equal to k. If ths s held, both members of (3-1) ca be premultpled by 1 ˆ 1 1 XX XX XX Xy XX : wth whch the expresso of the vector of least square estmators, or more precsely, the vector of ordary least square estmators (OLS), s obtaed because XX 1 XX I. Therefore, the soluto s the followg: ˆ 1 ˆ ˆ ˆk XX 1 Xy (3-3) Sce the matrx of secod dervatves, XX, s a postve defte matrx, the cocluso s that S presets a mmum ˆ. 3.. Iterpretato of the coeffcets A ˆ coeffcet measures the partal effect of the regressor x o y holdg the other regressors fxed. We wll see ext the meag of ths expresso. The ftted model for observato s gve by y ˆ ˆ x ˆ x ˆ x ˆ x (3-4) ˆ 1 3 3 k k Now, let us cosder the ftted model for observato h whch the values of the regressors ad, cosequetly, y wll have chaged wth respect to (3-4): y ˆ ˆ x ˆ x ˆ x ˆ x (3-5) ˆh 1 h 3 3 h h k kh Subtractg (3-5) from (3-4), we have y ˆ x ˆ x ˆ x ˆ x (3-6) ˆ 3 3 k k ˆ ˆ ˆ,,,. where y y yh x x xh x3 x3 x3h xk xk xkh The prevous expresso captures the varato of ŷ due to the chages all regressors. If oly x chages, we wll have y ˆ x (3-7) ˆ If x k creases oe ut, we wll have yˆ ˆ for x 1 (3-8) 6
Cosequetly, the coeffcet ˆ measures the chage y whe x creases 1 ut, holdg the regressors x, x3,, x 1, x 1,, xk fxed. It s very mportat to take to accout ths ceters parbus clause whe terpretg the coeffcet. Ths terpretato s ot vald, of course, for the tercept. EXAMPLE 3.1 Quatfyg the fluece of age ad wage o abseteesm the frm Bueosares Bueosares s a frm devoted to maufacturg fas, havg had relatvely acceptable results recet years. The maagers cosder that these would have bee better f the abseteesm the compay were ot so hgh. For ths purpose, the followg model s proposed: abset 1 age 3teure 4wage u where abset s measured days per year; wage thousads of euros per year; teure years the frm ad age s expressed years. Usg a sample of sze 48 (fle abset), the followg equato has bee estmated: abset = 14.413-0.096age -0.078teure- 0.036 wage (1.603) (0.048) (0.067) (0.007) R =0.694 =48 The terpretato of ˆ s the followg: holdg fxed teure ad wage, f age creases by oe year, worker abseteesm wll be reduced by 0.096 days per year. The terpretato of 3 ˆ s as follows: holdg fxed the age ad wage, f the teure creases by oe year, worker abseteesm wll be reduced by 0.078 days per year. Fally, the terpretato of 4 ˆ s the followg: holdg fxed the age ad teure, f the wage creases by 1000 euros per year, worker abseteesm wll be reduced by 0.036 days per year. EXAMPLE 3. Demad for hotel servces The followg model s formulated to expla the demad for hotel servces: l( hostel) b1 bl( c) b3hhsze u + + + (3-9) where hostel s spedg o hotel servces, c s dsposable come, both of whch are expressed euros per moth. The varable hhsze s the umber of household members. The estmated equato wth a sample of 40 households, usg fle hostel, s the followg: l( hostel ) - 7.36 + 4.44l( c )-0.53hhsze R =0.738 =40 As the results show, hotel servces are a luxury good. Thus, the demad/come elastcty for ths good s very hgh (4.44), whch s typcal of luxury goods. Ths meas that f come creases by 1%, spedg o hotel servces creases by 4.44%, holdg fxed the sze of the household. O the other had, f the household sze creases by oe member, the spedg o hotel servces wll decrease by 5%. EXAMPLE 3.3 A hedoc regresso for cars The hedoc model of prce measuremet s based o the assumpto that the value of a good s derved from the value of ts characterstcs. Thus, the prce of a car wll therefore deped o the value the buyer places o both qualtatve (e.g. automatc gear, power, desel, asssted steerg, ar codtog), ad quattatve attrbutes (e.g. fuel cosumpto, weght, performace dsplacemet, etc.). The data set for ths exercse s fle hedcarsp (hedoc car prce for Spa) ad covers years 004 ad 005. A frst model based oly o quattatve attrbutes s the followg: l( prce) 1volume 3fueleff u where volume s legth wdth heght m 3 ad fueleff s the lters per 100 km/horsepower rato expressed as a percetage. The estmated equato wth a sample of 14 observatos s the followg: l( prce ) 4.97 + 0.0956volume -0.1608 fueleff 7
R =0.765 =14 The terpretato of ˆ ad 3 ˆ s the followg. Holdg fxed fueleff, f volume creases by 1 m 3, the prce of a car wll rse by 9.56%. Holdg fxed volume, f the rato lters per 100 km/horsepower creases by 1 percetage pot, the prce of a car prce wll fall by 16.08%. EXAMPLE 3.4 Sales ad advertsg: the case of Lyda E. Pkham A model wth tme seres data s estmated order to measure the effect of advertsg expeses, realzed over dfferet tme perods, o curret sales. Deotg by V t ad P t sales ad advertsg expedtures, made at tme t, the model proposed tally to expla sales, as a fucto of curret ad past advertsg expeses s as follows: V P P P u (3-30) t 1 t t1 3 t t I the above expresso the dots dcate that past expedture o advertsg cotues to have a defte fluece, although t s assumed that wth a decreasg mpact o sales. The above model s ot operatoal gve that t has a defte umber of coeffcets. Two approaches ca be adopted order to solve the problem. The frst approach s to fx a pror the maxmum umber of perods durg whch advertsg effects o sales are mataed. I the secod approach, the coeffcets behave accordg to some law whch determes ther value based o a small umber of parameters, also allowg further smplfcato. I the frst approach the problem that arses s that, geeral, there are o precse crtera or suffcet formato to fx a pror the maxmum umber of perods. For ths reaso, we shall look at a specal case of the secod approach that s terestg due to the plausblty of the assumpto ad easy applcato. Specfcally, we wll cosder the case whch the coeffcets decrease geometrcally as we move backward tme accordg to the followg scheme: 1 0 1 (3-31) The above trasformato s called Koyck trasformato, as t was ths author who 1954 troduced scheme (3-31) for the study of vestmet Substtutg (3-31) (3-30), we obta V P P P u (3-3) t 1 t 1 t1 1 t t The above model stll has fte terms, but oly three parameters ad ca also be smplfed. Ideed, f we express equato (3-3) for perod t-1 ad multply both sdes by we obta V P P P u (3-33) 3 t1 1 t1 1 t 1 t3 t1 Subtractg (3-33) from (3-3), ad takg to accout factors ted to 0 as teds to fty, the result s the followg: V (1 ) P V u u (3-34) t 1 t t1 t t1 The model has bee smplfed so that t oly has three regressors, although, cotrast, t has moved to a compoud dsturbace term. Before seeg the applcato of ths model, we wll aalyze the sgfcace of the coeffcet ad the durato of the effects of advertsg expedtures o sales. The parameter s the decay rate of the effects of advertsg expedtures o curret ad future sales. The cumulatve effects that the advertsg expedture of oe moetary ut have o sales after m perods are gve by (3-35) (1 3 m ) 1 To calculate the cumulatve sum of effects, gve (3-35), we ote that ths expresso s the sum of the terms of a geometrc progresso 1, whch ca be expressed as follows: 1 Deotg by a p, a u ad r the frst term, the last term ad the rght respectvely, the sum of the terms of a coverget geometrc progresso s gve by 8
(1 m ) 1 (3-36) 1 Whe m teds to fty, the the sum of the cumulatve effects s gve by 1 (3-37) 1 A terestg pot s to determe how may perods of tme are requred to obta the p% (e.g., 50%) of the total effect. Deotg by h the umber of perods requred to obta ths percetage, we have h 1(1 ) Effect h perods 1 h p 1 Total effect 1 1 (3-38) Settg p, h ca be calculated accordg to (3-38). Solvg for h ths expresso, the followg s obtaed l(1 p) h (3-39) l Ths model was used by Krsta S. Palda hs doctoral thess publshed 1964, ettled The Measuremet of Cumulatve Advertsg Effects, to aalyze the cumulatve effects of advertsg expedtures the case of the compay Lyda E. Pkham. Ths case has bee the bass for research o the effects of advertsg expedtures. We wll see below some features of ths case: 1) The Lyda E. Pkham Medce Compay maufactured a herbal extract dluted a alcohol soluto. Ths product was orgally aouced as a aalgesc ad also as a remedy for a wde varety of dseases. ) I geeral, dfferet types of products there s ofte competto amog dfferet brads, as the paradgmatc case of Coca-Cola ad Peps-Cola. Whe ths occurs, the behavor of the ma compettors s take to accout whe aalyzg the effects of advertsg expedture. Lyda E. Pkham had the advatage of havg o compettors, actg as a moopolst practce ts product le. 3) Aother feature of the Lyda E. Pkham case was that most of the dstrbuto costs were allocated to advertsg because the compay had o commercal agets, wth the relatoshp betwee advertsg expeses ad sales beg very hgh. 4) The product was affected by dfferet avatars. Thus, 1914 the Food ad Drug Admstrato (Uted States agecy establshed cotrols for food ad medces) accused the frm of msleadg advertsg ad so they had to chage ther advertsg messages. Also, the Iteral Reveue (IRS) threateed to apply a tax o alcohol sce the alcohol cotet of the product was 18%. For all these reasos there were chages the presetato ad cotet durg the perod 1915-195. I 195 the Food ad Drug Admstrato baed the product from beg aouced as medce, havg to be dstrbuted as a toc drk. I the perod 196-1940 spedg o advertsg was sgfcatly creased ad shortly after the sales of the product decled. The estmato of the model (3-34) wth data from 1907 to 1960, usg fle pkham, s the followg: sales t = 138.7 + 0.388advexp + 0.7593salest - 1 R =0.877 =53 The sum of the cumulatve effects of advertsg expedtures o sales s calculated by the formula (3-37): a p a 1 r u 9
ˆ 1 0.388 1.3660 1 ˆ 1 0.7593 Accordg to ths result, every addtoal dollar spet o advertsg produces a accumulated total sale of 1,366 uts. Sce t s mportat ot oly to determe the overall effect, but also how log the effect lasts, we wll ow aswer the followg questo: how may perods of tme are requred to reach half of the total effects? Applyg the formula (3-39) for the case of p = 0.5, the followg result s obtaed: ˆ(0.5) l(1 0.5) h.517 l(0.7593) 3..3 Algebrac mplcatos of the estmato The algebrac mplcatos of the estmato are derved exclusvely from the applcato of the OLS method to the model of multple lear regresso: 1. The sum of the OLS resduals s equal to 0: uˆ 0 (3-40) 1 From the defto of resdual uˆ y yˆ y ˆ ˆ x ˆ x 1,,, (3-41) 1 k k If we add for the observatos, the ˆ ˆ ˆ uˆ y 1x kxk 1 1 1 1 (3-4) O the other had, the frst equato of the system of ormal equatos (3-0) s y 1 x kxk 1 1 1 ˆ ˆ ˆ 0 (3-43) If we compare (3-4) ad (3-43), we coclude that (3-40) holds. Note that, f (3-40) holds, t mples that y yˆ (3-44) 1 1 ad, dvdg (3-40) ad (3-44) by, we obta uˆ 0 y yˆ (3-45). The OLS hyperplae always goes through the pot of the sample meas yx,,, xk. By dvdg equato (3-43) by we have: y ˆ ˆ 1x ˆk x k (3-46) 3. The sample cross product betwee each oe of the regressors ad the OLS resduals s zero 10
x ˆ u = 0,3,, k (3-47) 1 Usg the last k ormal equatos (3-0) ad takg to accout that by defto u ˆ ˆ ˆ ˆ ˆí y 1x 3x3 k x k, we ca see that 1 1 1 ux ˆ 0 ux ˆ 0 3 ux ˆ 0 k (3-48) 4. The sample cross product betwee the ftted values ( ŷ ) ad the OLS resduals s zero. yu ˆˆ í 0 (3-49) 1 Takg to accout (3-40) ad (3-48), we obta yˆˆ u ( ˆ ˆ x ˆ x ) uˆ ˆ uˆ ˆ x uˆ ˆ x uˆ í 1 k k í 1 í í k k í 1 1 1 1 1 ˆ 0 ˆ 0 ˆ 00 1 k (3-50) 3.3 Assumptos ad statstcal propertes of the OLS estmators Before studyg the statstcal propertes of the OLS estmators the multple lear regresso model, we eed to formulate a set of statstcal assumptos. Specfcally, the set of assumptos that we wll formulate are called classcal lear model (CLM) assumptos. It s mportat to ote that CLM assumptos are smple, ad that the OLS estmators have, uder these assumptos, very good propertes. 3.3.1 Statstcal assumptos of the CLM multple lear regresso) a) Assumpto o the fuctoal form 1) The relatoshp betwee the regressad, the regressors ad the dsturbace s lear the parameters: y x x u (3-51) 1 k k+ or, alteratvely, for all the observatos, y=xβ +u (3-5) b) Assumptos o the regressors ) The values of x, x 3, xk are fxed repeated samplg, or the matrx X s fxed repeated samplg: 11
Ths s a strog assumpto the case of the socal sceces where, geeral, t s ot possble to expermet. A alteratve assumpto ca be formulated as follows: *) The regressors x, x 3,, xk are dstrbuted depedetly of the radom dsturbace. Formulated aother way, X s dstrbuted depedetly of the vector of radom dsturbaces, whch mples that E( Xu)=0 As we sad chapter, we wll adopt assumpto ). 3) The matrx of regressors, X, does ot cota dsturbaces of measuremet 4) The matrx of regressors, X, has rak k: ( X ) k (3-53) Recall that the matrx of regressors cotas k colums, correspodg to the k regressors the model, ad rows, correspodg to the umber of observatos. Ths assumpto has two mplcatos: 1. The umber of observatos,, must be equal to or greater tha the umber of regressors, k. Itutvely, to estmate k parameters, we eed at least k observatos.. Each regressor must be learly depedet, whch mples that a exact lear relatoshp amog ay subgroup of regressors caot exst. If a depedet varable s a exact lear combato of other depedet varables, the there s perfect multcollearty, ad the model caot be estmated. If a approxmate lear relatoshp exsts, the estmatos of the parameters ca be obtaed, although the relablty of such estmatos would be affected. I ths case, there s o-perfect multcollearty. c) Assumpto o the parameters 5) The parameters 1,, 3,, k are costat, or s a costat vector. d) Assumptos o the dsturbaces 6) The dsturbaces have zero mea, Eu ( ) 0, 1,,3,, or E( u) 0 (3-54) 7) The dsturbaces have a costat varace (homoskedastcty assumpto): var u (3-55) ( ) 1,, 8) The dsturbaces wth dfferet subscrpts are ot correlated wth each other (o autocorrelato assumpto): Euu ( ) 0 (3-56) The formulato of homoskedastcty ad o autocorrelato assumptos allows us to specfy the covarace matrx of the dsturbace vector: 1
E ue( u) ue( u) E u0u0 E uu u1 u1 uu 1 uu 1 u uu 1 u uu E u1 u u E (3-57) u uu 1 uu u Eu ( 1) Euu ( 1 ) Euu ( 1 ) 0 0 Euu ( 1) Eu ( ) Euu ( ) 0 0 Euu ( 1) E( uu ) Eu ( ) 0 0 I order to get to the last equalty, t has bee take to accout that the varaces of each oe of the elemets of the vector s costat ad equal to accordace wth (3-55) ad the covaraces betwee each par of elemets s 0 accordace wth (3-56). The prevous result ca be expressed sythetc form: E( ) uu I (3-58) The matrx gve (3-58) s deomated scalar matrx, sce t s a scalar (, ths case) multpled by the detty matrx. 9) The dsturbace u s ormally dstrbuted Takg to accout assumptos 6 to 9, we have u NID, or ~ (0 ) 1,,, where NID stads for ormally depedetly dstrbuted. u~ N( 0, I ) (3-59) 3.3. Statstcal propertes of the OLS estmator Uder the above assumptos of the CLM, the OLS estmators possess good propertes. I the proofs of ths secto, assumptos 3, 4 ad 5 wll mplctly be used. Learty ad ubasedess of the OLS estmator Now, we are gog to prove that the OLS estmator s learly ubased. Frst, we express ˆβ as a fucto of the vector u, usg assumpto 1, accordg to (3-5): ˆ -1-1 -1 β = XX Xy = XX X Xβ +u =β +XX Xu (3-60) The OLS estmator ca be expressed ths way so that the property of learty s clearer: where -1 ˆ -1 β = β +XX Xu=β +Au (3-61) A= XX X s fxed uder assumpto. Thus ˆβ s a lear fucto of u ad, cosequetly, t s a lear estmator. Takg expectatos (3-60) ad usg assumpto 6, we obta 13
-1 E Eβ ˆ = β + XX X u =β (3-6) Therefore, ˆβ s a ubased estmator. Varace of the OLS estmators I order to calculate the covarace matrx of ˆβ assumptos 7 ad 8 are eeded, addto to the frst sx assumptos: var( β ˆ) = Eβˆ E( βˆ) βˆ E( β ˆ) = Eβˆ βˆβˆ βˆ = E XX XuuX XX = XX X uu X XX E( ) -1 E( -1 ) -1 = XX X I X XX = XX -1-1 -1-1 (3-63) I the thrd step of the above proof t s take to accout that, accordg to -1 (3-60), βˆ β= XX Xu. Assumpto s take to accout the fourth step. Fally, assumptos 7 ad 8 are used the last step. ˆ -1 var( β) XX s the covarace matrx of the vector ˆβ. I ths Therefore, covarace matrx, the varace of each elemet ˆ appears o the ma dagoal, whle the covaraces betwee each par of elemets are outsde of the ma dagoal. Specfcally, the varace of ˆ (for =,3,,k) s equal to multpled by the correspodg elemet of the ma dagoal of XX -1. After operatg, the varace of ˆ ca be expressed as var( ˆ ) S (1 R ) (3-64) where R s the R-squared from regressg x o all other x s, s the sample sze ad S s the sample varace of the regressor X. Formula (3-64) s vald for all slope coeffcets, but ot for the tercept The square root of (3-64) s called the stadard devato of ˆ : sd( ˆ ) S (1 R ) (3-65) OLS estmators are BLUE Uder assumptos 1 through 8 of the CLM, whch are called Gauss-Markov assumptos, the OLS estmators s the Best Lear Ubased Estmators (BLUE). The Gauss Markov theorem states that the OLS estmator s the best estmator wth the class of lear ubased estmators. I ths cotext, best meas that t s a estmator wth the smallest varace for a gve sample sze. Let us ow compare the varace of a elemet of ˆβ (ˆ ), wth ay other estmator that s lear (so 14
wy) ad ubased (so the weghts, w, must satsfy some restrctos). The 1 property of ˆ beg a BLUE estmator has the followg mplcatos whe comparg ts varace wth the varace of : ˆ 1) The varace of the coeffcet s greater tha, or equal to, the varace of obtaed by OLS: var( ) var( ˆ ) 1,,3,, (3-66) k ) The varace of ay lear combato of s s greater tha, or equal to, the varace of the correspodg lear combato of ˆ s. I appedx 3.1 the proof of the theorem of Gauss-Markov ca be see. Estmator of the dsturbace varace Takg to accout the system of ormal equatos (3-0), f we kow k of the resduals, we ca get the other k resduals by usg the restrctos mposed by that system the resduals. For example, the frst ormal equato allows us to obta the value of u ˆ as a fucto of the remag resduals: uˆ ˆ ˆ ˆ u u u 1 1 Thus, there are oly k degrees of freedom the OLS resduals, as opposed to degrees of freedom the dsturbaces. Remember that the degree of freedom s defed as the dfferece betwee the umber of observatos ad the umber of parameters estmated. The ubased estmator of s adusted take to accout the degree of freedom: uˆ 1 ˆ k Uder assumptos 1 to 8, we obta ˆ (3-67) E( ) (3-68) See appedx 3. for the proof. The square root of (3-67), ˆ s called stadard error of the regresso ad s a estmator of. Estmators of the varaces of ˆβ ad the slope coeffcet ˆ The estmator of the covarace matrx of ˆβ s gve by 15
Var( ˆ ) ˆ XX 1 var( ˆ 1) Cov( ˆ 1, ˆ ) Cov( ˆ 1, ˆ ˆ ˆ ) Cov( 1, ) k ˆ ˆ ˆ ˆ ˆ Cov( ˆ ˆ, 1) var( ) Cov(, ) Cov(, ) k ˆ ˆ ˆ ˆ ˆ Cov(, ˆ ˆ 1) Cov(, ) var( ) Cov(, k) ˆ ˆ Cov(, ˆ ˆ ˆ k 1) Cov( k, ) Cov( k, ˆ ) var( ˆ k) (3-69) The varace of the slope coeffcet ˆ, gve (3-64), s a fucto of the ukow parameter. Whe varace of ˆ s obtaed: s substtuted by ts estmator ˆ, a estmator of the var( ˆ ) S ˆ (1 R ) (3-70) Accordg to the prevous expresso, the estmator of the varace ˆ affected by the followg factors: a) The greater ˆ, the greater the varace of the estmator. Ths s ot at all surprsg: more ose the equato - a larger ˆ - makes t more dffcult to estmate accurately the partal effect of ay x s o y. (See fgure 3.1). b) As sample sze creases, the varace of the estmator s reduced. c) The smaller the sample varace of a regressor, the greater the varace of the correspodg coeffcet. Everythg else beg equal, for estmatg we prefer to have as much sample varato x as possble, whch s llustrated fgure 3.. As you ca see, there are may hypothetcal les that could ft the data whe the sample varace of x ( S ) s small, whch ca be see part a) of the fgure. I ay case, assumpto 4 does ot allow S beg equal to 0. d) The hgher R, (.e., the hgher s the correlato of regressor wth the rest of the regressors), the greater the varace of ˆ. s 16
y y x x a) ŝ bg b) ŝ small FIGURE 3.1. Ifluece of ŝ o the estmator of the varace. y y a) FIGURE 3.. Ifluece of x S small b) S bg S o the estmator of the varace. x The square root of (3-70) s called the stadard error of ˆ : se( ˆ ) ˆ S (1 R ) (3-71) Other propertes of the OLS estmators Uder 1 through 6 CLM assumptos, the OLS estmator ˆβ s cosstet, as ca be see appedx 3.3, asymptotcally ormally dstrbuted ad also asymptotcally effcet wth the class of the cosstet ad asymptotcally ormal estmators. Uder 1 through 9 CLM assumptos, the OLS estmator s also the maxmum lkelhood estmator (ML), as ca be see appedx 3.4, ad the mmum varace ubased estmator (MVUE). Ths meas that the OLS estmator has the smallest varace amog all ubased, lear o o lear, estmators. 3.4 More o fuctoal forms I ths secto we wll exame two topcs o fuctoal forms: use of atural logs models ad polyomal fuctos. 3.4.1 Use of logarthms the ecoometrc models Some varables are ofte used log form. Ths s the case of varables moetary terms whch are geerally postve or varables wth hgh values such as populato. Usg models wth log trasformatos also has advatages, oe of whch s that coeffcets have appealg terpretatos (elastcty or sem-elastcty). Aother advatage s the varace of slopes to scale chages the varables. Takg logs s also very useful because t arrows the rage of varables, whch makes estmates less sestve to extreme observatos o the depedet or the depedet varables. The CLM assumptos are satsfed more ofte models usg l(y) as a regressad tha 17
models usg y wthout ay trasformato. Thus, the codtoal dstrbuto of y s frequetly heteroskedastc, whle l(y) ca be homoskedastc. Oe lmtato of the log trasformato s that t caot be used whe the orgal varable takes zero or egatve values. O the other had, varables measured years ad varables that are a proporto or a percetage, are ofte used level (or orgal) form. 3.4. Polyomal fuctos The polyomal fuctos have bee extesvely used ecoometrc research. Whe there are oly the regressors correspodg to a polyomal fucto we have a polyomal model. The geeral k th degree polyomal model may be wrtte as Quadratc fuctos y x x x u (3-7) 1 3 + k k + A terestg case of polyomal fuctos s the quadratc fucto, whch s a secod-degree polyomal fucto. Whe there are oly regressors correspodg to the quadratc fucto, we have a quadratc model: y x x u (3-73) 1 3 + Quadratc fuctos are used qute ofte appled ecoomcs to capture decreasg or creasg margal effects. It s mportat to remark that, such a case, does ot measure the chage y wth respect to x because t makes o sese to hold x fxed whle chagg x. The margal effect of x o y, whch depeds learly o the value of x, s the followg: dy me 3x (3-74) dx I a partcular applcato ths margal effect would be evaluated at specfc values of x. If ad 3 have opposte sgs the turg pot wll be at * x (3-75) 3 If >0 ad 3 <0, the the margal effect of x o y s postve at frst, but t wll be egatve for values of x greater tha x *. If <0 ad 3 >0, ths margal effect s * egatve at frst, but t wll be postve for values of x greater tha x. Example 3.5 Salary ad teure Usg the data ceosal to study the type of relato betwee the salary of the Chef Executve Offcers (CEOSs) USA corporatos ad the umber of years the compay as CEO (ceote), the followg model was estmated: l( salary) 6.460.0006 profts 0.0440ceote 0.001ceote (0.086) (0.0001) (0.0156) (0.0005) R =0.1976 =177 where compay profts are mllos of dollars ad salary s aual compesato thousads of dollars. The margal effect ceote o salary expressed percetage s the followg: me salary / ceote % 4.40 0.1ceote 18
Thus, f a CEO wth 10 years a compay speds oe more year that compay, ther salary wll crease by %. Equatg to zero the prevous expresso ad solvg for ceote, we fd that the maxmum effect of teure as CEO o salary s reached by 18 years. That s, utl 18 years the margal effect of CEO teure o the salary s postve. O the cotrary, from 18 years owards ths margal effect s egatve. Cubc fuctos Aother terestg case s the cubc fucto, or thrd-degree polyomal fucto. If the model there are oly regressors correspodg to the cubc fucto, we have a cubc model: y x x x u (3-76) 3 1 3 4 Cubc models are used qute ofte appled ecoomcs to capture decreasg or creasg margal effects, partcularly the cost fuctos. The margal effect (me) of x o y, whch depeds o x a quadratc form, wll be the followg: dy me x x dx The mmum of me wll occur where dme 364x 0 (3-78) dx Therefore, 3 34 (3-77) me m 3 (3-79) 3 I a cubc model of a cost fucto, the restrcto 3 3 4 must be met to guaratee that the mmum margal cost s postve. Other restrctos that a cost fucto must satsfy are as follows: 1,, ad 4 >0; ad 3 <0 Example 3.6 The margal effect a cost fucto Usg the data o 11 pulp mll frms (fle costfuc) to study the cost fucto, the followg model was estmated: 3 cost 9.16.316output 0.0914output 0.0013output (1.60) (0.167) (0.0081) (0.000086) R =0.9984 =11 where output s the producto of pulp thousads of tos ad cost s the total cost mllos of euros The margal cost s the followg: marcost.316 0.0914output 3 0.0013output Thus, f a frm wth a producto of 30 thousad tos of pulp creases the pulp producto by oe thousad tos, the cost wll crease by 0.754 mllo of euros. Calculatg the mmum of the above expresso ad solvg for output, we fd that the mmum margal cost s equal to a producto of 3. thousad tos of pulp. 3.5 Goodess-of-ft ad selecto of regressors. Oce least squares have bee appled, t s very useful to have some measure of the goodess of ft betwee the model ad the data. I the evet that several alteratve models have bee estmated, measures of the goodess of ft could be used to select the most approprate model. 4 19
I ecoometrc lterature there are umerous measures of goodess of ft. The most popular s the coeffcet of determato, whch s desgated by R or R- squared, ad the adusted coeffcet of determato, whch s desgated R or adusted R-squared. Gve that these measures have some lmtatos, the Akake Iformato Crtero (AIC) ad Schwarz Crtero (SC) wll also be referred to later o. 3.5.1 Coeffcet of determato As we saw chapter, the coeffcet of determato s based o the followg breakdow: TSS ESS RSS (3-80) where TSS s the total sum of squares, ESS s the explaed sum of squares ad RSS s the resdual sum of squares. Based o ths breakdow, the coeffcet of determato s defed as: ESS R (3-81) TSS Alteratvely, ad a equvalet maer, the coeffcet of determato ca be defed as RSS R (3-8) TSS The extreme values of the coeffcet of determato are: 0, whe the explaed varace s zero, ad 1, whe the resdual varace s zero; that s, whe the ft s perfect. Therefore, (3-83) 0 R 1 A small R mples that the dsturbace varace ( ) s large relatve to the varace of y, whch meas that s ot estmated wth precso. But remember that a large dsturbace varace ca be offset by a large sample sze. Thus, f s large eough, we may be able to estmate the coeffcets wth precso eve though we have ot cotrolled for may uobserved factors. To terpret the coeffcet of determato properly, the followg caveats should be take to accout: a) As ew explaatory varables are added, the coeffcet of determato creases ts value or, at least, keeps the same value. Ths happes eve though the varable (or varables) added have o relato to the edogeous varable. Thus, we ca always verfy that R ³ R - (3-84) 1 R - R where 1the R s squared a model wth -1 regressors, ad s the R squared a model wth a addtoal regressor. That s to say, f we add varables to a gve model, R wll ever decrease, eve f these varables do ot have a sgfcat fluece. b) If the model has o tercept, the coeffcet of determato does ot have a clear terpretato because the decomposto gve (3-80) s ot fulflled. I addto, the two forms of calculato metoed - (3-81) ad (3-8) - geerally lead to dfferet results, whch some cases may fall outsde the terval [0, 1]. 0
c) The coeffcet of determato caot be used to compare models whch the fuctoal form of the edogeous varable s dfferet. For example, R caot be appled to compare two models whch the regressad s the orgal varable, y, ad l(y) respectvely. 3.5. Adusted R-Squared To overcome oe of the lmtatos of the R, we ca adust t a way that takes to accout the umber of varables cluded a gve model. To see how the usual R mght be adusted, t s useful to wrte t as RSS / R = 1- (3-85) TSS / where, the secod term of the rght-had sde, the resdual varace s dvded by the varace of the regressad. The R, as t s defed (3-85), s a sample measure. Now, f we wat a populato measure, we ca defe the populato R as R POP (3-86) u 1 y However, we have better estmates for these varaces, u ad y, tha the oes used the (3-85). So, let us use ubased estmates for these varaces /( ) 1 R = 1- SCR -k = 1 -(1-R ) - (3-87) SCT /( -1) -k Ths measure s called the adusted R squared, or R.The prmary attractveess of R s that t mposes a pealty for addg addtoal regressors to a model. If a regressor s added to the model the RSS decreases, or at least s equal. O the other had, the degrees of freedom of the regresso k always decrease. R ca go up or dow whe a ew regressor s added to the model. That s to say: R ³ R - or 1 R R - (3-88) 1 A terestg algebrac fact s that f we add a ew regressor to a model, R creases f, ad oly f, the t statstc, whch we wll exame chapter 4, o the ew regressor s greater tha 1 absolute value. Thus we see mmedately that R could be used to decde whether a certa addtoal regressor must be cluded the model. The R has a upper boud that s equal to 1, but t does ot strctly have a lower boud sce t ca take egatve values. The observatos b) ad c) made to the R squared rema vald for the adusted R squared. 3.5.3 Akake formato crtero (AIC) ad Schwarz crtero (SC) These two crtera- Akake formato crtero (AIC) ad Schwarz Crtero (SC) - have a very smlar structure. For ths reaso, they wll be revewed together. The AIC statstc, proposed by Akake (1974) ad based o formato theory, has the followg expresso: 1
l k AIC =- + (3-89) where l s the log lkelhood fucto (assumg ormally dstrbuted dsturbaces) evaluated at the estmated values of the coeffcets. The SC statstc, proposed by Schwarz (1978), has the followg expresso: l kl( ) SC =- + (3-90) The AIC ad SC statstcs, ulke the coeffcets of determato (R ad R ), are better the lower ther values are. It s mportat to remark that the AIC ad SC statstcs are ot bouded ulke R. a) The AIC ad SC statstcs pealze the troducto of ew regressors. I the case of the AIC, as ca be see the secod term of the rght had sde of (3-89), the umber of regressors k appears the umerator. Therefore, the growth of k wll crease the value of AIC ad cosequetly worse the goodess of ft, f that s ot offset by a suffcet growth of the log lkelhood. I the case the SC, as ca be see the secod term of the rght had sde of (3-90), the umerator s kl(). For >7, the followg happes: kl()>k. Therefore, SC mposes a larger pealty for addtoal regressors tha AIC whe the sample sze s greater tha seve. b) The AIC ad SC statstcs ca be appled to statstcal models wthout tercept. c) The AIC ad SC statstcs are ot relatve measures as are the coeffcets of determato. Therefore, ther magtude, tself, offers o formato. d) The AIC ad SC statstcs ca be appled to compare models whch edogeous varables have dfferet fuctoal forms. I partcular, we wll compare two models whch the regressads are y ad l(y). Whe the regressad s y, the formula (3-89) s appled the AIC case, or (3-90) the SC case. Whe the regressad s l(y), ad also whe we wat to carry out a comparso wth aother model whch the regressad s y, we must correct these statstcs the followg way: AIC = AIC + l( Y ) (3-91) C SC = SC + l( Y ) (3-9) C where AIC C ad SC C are the corrected statstcs, ad AIC ad SC are the statstcs suppled by ay ecoometrc package such as the E-vews. Example 3.7 Selecto of the best model To aalyze the determats of expedtures o dary the followg alteratve models have bee cosdered: 1) dary 1c u ) dary l( ) 1 c u 3) dary 1c 3puder5 u 4) dary c 3puder5 u 5) dary 1 c 3hhsze u 6) l( dary) 1c u 7) l( dary) 1 c 3puder5 u l( dary) c puder5 u 8) 3
where c s dsposable come of household, hhsze s the umber of household members ad puder5 s the proporto of chldre uder fve the household. Usg a sample of 40 households (fle demad), ad takg to accout that l( dary ) =.3719, the goodess of ft statstcs obtaed for the eght models appear table 1. I partcular, the AIC corrected for model 6) has bee calculated as follows: AICC = AIC + l( Y ) = 0.794 +.3719=5.03 Coclusos a) The R-squared ca be oly used to compare the followg pars of models: 1) wth ), ad 3) wth 5). b) The adusted R-squared ca oly be used to compare model 1) wth ), 3) ad 5); ad 6) wth 7. c) The best model out of the eght s model 7) accordg to AIC ad SC. TABLE 3.1. Measures of goodess of ft for eght models. Model umber 1 3 4 5 6 7 8 Regressad dary dary dary dary dary l(dary) l(dary) l(dary) Regressors tercept c tercept l(c) tercept c puder5 c puder5 tercept Ic househsze tercept c tercept c puder5 c puder5 R-squared 0.4584 0.4567 0.5599 0.5531 0.4598 0.4978 0.5986-0.6813 Adusted R-squared 0.4441 0.444 0.5361 0.5413 0.4306 0.4846 0.5769-0.755 Akake formato crtero 5.374 5.404 5.0798 5.045 5.847 0.794 0.105 1.4877 Schwarz crtero 5.319 5.349 5.065 5.196 5.4113 0.3638 0.319 1.571 Corrected Akake formato crtero Corrected Schwarz crtero 5.03 4.8490 6.314 5.1076 4.9756 6.3159 3
Exercses Exercse 3.1 Cosder the lear regresso model y=xβ +u, where X s a matrx 50 5. Aswer the followg questos, ustfyg your aswers: a) What are the dmesos of the vectors y, β, u? b) How may equatos are there the system of ormal equatos XXβ ˆ =Xy? c) What codtos are eeded order to obta ˆβ? Exercse 3. Gve the model ad the followg data: y =β 1 +β x +β 3 x 3 +u y x x 3 10 1 0 5 3-1 3 4 0 43 5 1 58 7-1 6 8 0 67 10-1 71 10 a) Estmate β 1, β ad β 3 by OLS. b) Calculate the resdual sum of squares. c) Obta the resdual varace. d) Obta the varace explaed by the regresso. e) Obta the varace of the edogeous varable f) Calculate the coeffcet of determato. g) Obta a ubased estmato of σ. h) Estmate the varace of ˆ. To aswer these questos you ca use Excel. See exhbt 3.1 as a example. 4
Exhbt 3.1 1) Calculato of X X ad X y Explaato for X X a) Eter the matrces X ad X to the Excel: B5:K6 ad N:O11 b) You ca fd the product X XX by hghlghtg the cells where you wat to place the resultg matrx. c) Oce you have hghlghted the t resultg matrx, ad whle t s stll hghlghted, eter the followg formula:=mmult(b5:k6; N:O11) d) Whe the formula s etered, press the Ctrl key ad the Shft key smultaeously. The, holdg thesee two keys, press the Eter key too. ) Calculato of (X X) -1 a) Eter the matrx X X to the Excel: R5:S6 b) You ca fd the verse of matrx X X byy hghlghtgg the cells where you wat to place the resultg matrx (R5:S6) c) Oce you have hghlghted the resultg matrx, ad whle t s stll hghlghted, eter the followg formula:=minverse(r5:s6).. d) Whe the formula s etered, press the Ctrl key ad the Shft key smultaeously. The, holdg thesee two keys, press the Eter key too. 3) Calculato of vector ˆβ 4) Calculato of û ' uˆ u ˆ ad σ uu ˆˆ ' ' ' ' = yy- yy ˆˆ= - ˆ' ' yy β X Xˆ ' Xβ = yy-βˆ' ' Xy = R. 5-R. 6 = 953-883=70 s ˆ uu ˆˆ 70 = = = 8.6993-8 5) Calculato of covarace matrx of ˆβ ' var( ˆ β) = sˆ é - ê X X ù 1 æ 3.8696-0 = 8.69933 0.0370ö ë úû ç æ 33.664-0.315ö = çè -0.0370 0.0004 ø çè-0..315 0.003 ø ' 5
Exercse 3.3 The followg model was formulated to expla the aual sales (sales) of the maufacturers of household cleag products as a fucto of a relatve prce dex (rp) ad the advertsg expedtures (adv): sales 1rp 3adv u where the varable sales s expressed a thousad mllo euros ad rp s a relatve prce dex obtaed as a rato betwee the prces of each frm ad the prces of frm 1 of the sample; adv s the aual expedtures o advertsg ad promotoal campags ad meda dffuso, expressed mllos of euros. Data o te maufacturers of household cleag products appear the attached table. frm sales rp adv 1 10 100 300 8 110 400 3 7 130 600 4 6 100 100 5 13 80 300 6 6 80 100 7 1 90 600 8 7 10 00 9 9 10 400 10 15 90 700 Usg a excel spreadsheet, a) Estmate the parameters of the proposed model b) Estmate the covarace matrx. c) Calculate the coeffcet of determato. Note: I exhbt 3.1 the model sales 1rp u s estmated usg excel. Istructos are also cluded. Exercse 3.4 A researcher, who s developg a ecoometrc model to expla come, formulates the followg specfcato: c=α+βcos+γsave+u [1] where c s the household dsposable come, cos s the total cosumpto ad save s the total savgs of the household. The researcher dd ot take to accout that the above three magtudes are related by the detty c=cos+save [] The equvalece betwee the models [1] ad [] requres that, addto to the dsappearace of the dsturbace term, the model parameters [1] take the followg values: α =0, β =1, ad γ =1 If you estmate equato [1] wth the data for a gve coutry, ca you expect, geeral, that the estmates wll take the values ˆ 0, ˆ 1, ˆ 0? Please ustfy your aswer usg mathematcal otato. Exercse 3.5 A researcher proposes the followg ecoometrc model to expla toursm reveue (turtot) a gve coutry: turtot turmea umtur u 1 3 where turmea s the average expedture per tourst ad umtur s the total umber of toursts. 6
a) It s obvous that turtot, umtur ad turmea ad are also lked by the relatoshp turtot=turmea umtur. Wll ths somehow affect the estmato of the parameters of the proposed model? b) Is there a model wth aother fuctoal form volvg tghter restrctos o the parameters? If so, dcate t. c) What s your opo about usg the proposed model to expla the behavor of toursm reveue? Is t reasoable? Exercse 3.6 Let us suppose you have to estmate the model l( y) 1l( x) 3l( x3) 4l( x4) u usg the followg observatos: x x 3 x 4 3 1 4 10 5 4 4 1 3 9 3 6 3 5 5 1 What problems ca arse the estmato of ths model? Exercse 3.7 Aswer the followg questos: a) Expla the determato coeffcet (R ) ad the adusted determato coeffcet ( R ). What ca you use them for? Justfy your aswer. b) Gve the models l(y)=β 1 +β l(x)+u (1) l(y)=β 1 +β l(x)+β 3 l(z)+u () l(y)=β 1 +β l(z)+u (3) y=β 1 +β z+u (4) dcate what measure of goodess of ft s approprate to compare the followg pars of models: (1) - (), (1) - (3), ad (1) - (4). Expla your aswer. Exercse 3.8 Let us suppose that the followg model s estmated by OLS: l( y) 1l( x) 3l( z) u a) Ca least square resduals all be postve? Expla your aswer. b) Uder the assumpto of o autocorrelato of dsturbaces, are the OLS resduals depedet? Expla your aswer c) Assumg that the dsturbaces are ot ormally dstrbuted, wll the OLS estmators be ubased? Expla your aswer. Exercse 3.9 Cosder the lear regresso model yxu where y ad u are vectors 81, X s a matrx 83 ad s a vector 31. Also the followg formato s avalable: 0 0 XX 0 3 0 uu ˆˆ 0 0 3 Aswer the followg questos, by ustfyg your aswer: 7
a) Idcate the sample sze, the umber of regressors, the umber of parameters ad the degrees of freedom of the resdual sum of squares. b) Derve the covarace matrx of the vector ˆ, makg explct the assumptos used. Estmate the varaces of the estmators. c) Does the regresso have a tercept? What mplcatos does the aswer to ths questo have o the meag of R ths model? Exercse 3.10 Dscuss whether the followg statemets are true or false: a) I a lear regresso model, the sum of the resduals s zero. b) The coeffcet of determato ( R ) s always a good measure of the model s qualty. c) The least squares estmators are based. Exercse 3.11 The followg model s formulated to expla tme spet sleepg: sleep 1 totalwrk 3lesure u where sleep, totalwrk (pad ad upad work) ad lesure (tme ot devoted to sleep or work) are measured mutes per day. The estmated equato wth a sample of 1000 observatos, usg fle tmuse03, s the followg: sleep = 1440-1 total _ work - 1 lesure R =1.000 =1000 a) What do you thk about these results? b) What s the meag of the estmated tercept? Exercse 3.1 Usg a subsample of the Structural Survey of Wages (Ecuesta de estructura salaral) for Spa 006 (fle wage06sp), the followg model s estmated to expla wage: l( wage ) 1.565 0.0730 educ 0.0177 teure 0.0065 age R =0.337 =800 where educ (educato), teure (experece the frm) ad age are measured years ad wage euros per hour. a) What s the terpretato of coeffcets o educ, teure ad age? b) How may years does the age have to crease order to have a smlar effect to a crease of oe year educato, holdg fxed each case the other two regressors? c) Kowg that educ =10., teure =7. ad age =4.0, calculate the elastctes of wage wth respect to educ, teure ad age for these values, holdg fxed the others regressors. Do you cosder these elastctes to be hgh or low? Exercse 3.13 The followg equato descrbes the prce of housg terms of house bedrooms (umber of bedrooms), bathrms (umber of full bathrooms) ad lotsze (the lot sze of a property square feet): prce 1bedrooms 3bathrms 4lotsze u where prce s the prce of a house measured dollars. 8
Usg the data for the cty of Wdsor cotaed fle houseca, the followg model s estmated: prce 418 587 bedrooms 19750 bathrms 5.411 lotsze R =0.486 =546 a) What s the estmated crease prce for a house wth oe more bedroom ad oe more bathroom, holdg lotsze costat? b) What percetage of the varato prce s explaed otly by the umber of bedrooms, the umber of full bathrooms ad the lot sze? c) Fd the predcted sellg prce for a house of the sample wth bedrooms=3, bathrms= ad lotsze=3880. d) The actual sellg prce of the house c) was $66,000. Fd the resdual for ths house. Does the result suggest that the buyer uderpad or overpad for the house? Exercse 3.14 To exame the effects of a frm s performace o a CEO salary, the followg model was formulated: l( salary) 1roa 3l( sales) 4profts 5teure u where roa s the rato profts/assets expressed as a percetage ad teure s the umber of years as CEO (=0 f less tha 6 moths). Salares are expressed thousads of dollars, ad sales ad profts mllos of dollars. The fle ceoforbes has bee used for the estmato. Ths fle cotas data o 447 CEOs of Amerca's 500 largest corporatos. (5 of the 500 frms were excluded because of mssg data o oe or more varables. Apple Computer was also excluded sce Steve Jobs, the actg CEO of Apple 1999, receved o compesato durg ths perod.) Compay data come from Fortue magaze for 1999; CEO data come from Forbes magaze for 1999 too. The results obtaed were the followg: l( salary ) 4.641 0.0054 roa 0.893l( sales ) 0.0000564 profts 0.01 teure R =0.3 =447 a) Iterpret the coeffcet o the regressor roa b) Iterpret the coeffcet o the regressor l(sales). What s your opo about the magtude of the elastcty salary/sales? c) Iterpret the coeffcet o the regressor profts. d) What s the salary/profts elastcty at the sample mea ( salary =08 ad profts =700). Exercse 3.15 (Cotuato of exercse.1) Usg a dataset cosstg of 1,983 frms surveyed 006 (fle rdspa), the followg equato was estmated: rdtes =- 1.8168 + 0.148l( sales ) + 0.0110 exposal R = 0.048 =1983 where rdtes s the expedture o research ad developmet (R&D) as a percetage of sales, sales are measured mllos of euros, ad exposal s exports as a percetage of sales. a) Iterpret the coeffcet o l(sales). I partcular, f sales crease by 100%, what s the estmated percetage pot chage rdtes? Is ths a ecoomcally large effect? b) Iterpret the coeffcet o exposal. Is t ecoomcally large? 9
c) What percetage of the varato rdtes s explaed by sales ad exposal? d) What s the rdtes/sales elastcty for the sample mea ( rdtes =0.73 ad sales =63544960). Commet o the result. e) What s the rdtes/exposal elastcty for the sample mea ( rdtes =0.73 ad exposal =17.657). Commet o the result. Exercse 3.16 The followg hedoc regresso for cars (see example 3.3) s formulated: l( prce) 1cd 3hpweght 4fueleff u where cd s the cubc ch dsplacemet, hpweght s the rato horsepower/weght kg expressed as percetage ad fueleff s the rato lters per 100 km/horsepower expressed as a percetage. a) What are the probable sgs of β, β 3 ad β 4? Expla them. b) Estmate the model usg the fle hedcarsp ad wrte out the results equato form. c) Iterpret the coeffcet o the regressor cd. d) Iterpret the coeffcet o the regressor hpweght. e) To expad the model, add a regressor relatve to car sze, such as volume or weght. What happes f you add both of them? What s the relatoshp betwee weght ad volume? Exercse 3.17 The cocept of work covers a broad spectrum of possble actvtes the productve ecoomy. A mportat part of work s upad; t does ot pass through the market ad therefore has o prce. The most mportat upad work s housework (houswork) carred out maly by wome. I order to aalyze the factors that fluece housework, the followg model s formulated: houswork 1educ 3hhc 4age 5padwork u where educ s the years of educato attaed, hhc s the household come euros per moth. The varables houswork ad padwork are measured mutes per day. Use the data the fle tmuse03 to estmate the model. Ths fle cotas 1000 observatos correspodg to a radom subsample extracted from the tme use survey for Spa carred out 00-003. a) Whch sgs do you expect for β, β 3, β 4 ad β 5? Expla. b) Wrte out the results equato form? c) Do you thk there are relevat factors omtted the above equato? Expla. d) Iterpret the coeffcet o the regressors educ, hhc, age ad padwork. Exercse 3.18 (Cotuato of exercse.0) To expla the overall satsfacto of people (stsfglo), the followg model s formulated: stsfglo gpc lfexpec u 1 3 where gpc s the gross atoal come per capta expressed PPP 008 US dollar terms ad lfexpec s the lfe expectacy at brth,.e., the umber of years a ewbor fat could expect to lve. Whe a magtude s expressed PPP (purchasg power party) US dollar terms, a magtude s coverted to teratoal dollars usg PPP 30
rates. (A teratoal dollar has the same purchasg power as a US dollar the Uted States.) Use the fle HDR010 for the estmato of the model. a) What are the expected sgs for β ad β 3? Expla. b) What would be the average overall satsfacto for a coutry wth 80 years of lfe expectacy at brth ad a gross atoal come per capta of 30000 $ expressed PPP 008 US dollars? c) Iterpret the coeffcets o gpc ad lfexpe. d) Gve a coutry wth a lfe expectacy at brth equal to 50 years, what should be the gross atoal come per capta to obta a global satsfacto equal to fve? Exercse 3.19 (Cotuato exercse.4) Due to the problems arse the Keyesa cosumpto fucto, Brow troduced a ew regressor the fucto: cosumpto lagged a perod to reflect the persstece of cosumer habts. The formulato of the model s as follows cospc b + b cpc + b cospc - + u t 1 t 3 t 1 t As lagged cosumpto s cluded ths model, we have to dstgush betwee margal propesty to cosume the short term ad log term. The short-ru margal propesty s calculated the same way as the Keyesa cosumpto fucto. To calculate the log-term margal propesty t s ecessary to cosder equlbrum state wth o chages varables. Deotg by cospc e ad cpc e cosumpto ad come equlbrum, ad regardless of the radom dsturbace, the prevous model equlbrum s gve by cospc b + b cpc + b cospc e e e 1 3 The Brow cosumpto fucto was estmated wth data of the Spash ecoomy for the perod 1954-010 (fle cosumsp), obtag the followg results: cospc t 7.156 0.3965cpct 0.5771cospct 1 R =0.997 =56 a) Iterpret the coeffcet o cpc. I the terpretato, do you have to clude the clause "holdg fxed the other regressor? Justfy the aswer. b) Calculate the short-term elastcty for the sample meas ( cospc =8084, cpc =8896). c) Calculate the log-term elastcty for the sample meas. d) Dscuss the dfferece betwee the values obtaed for the two types of elastcty. Exercse 3.0 To expla the fluece of cetves ad expedtures advertsg o sales, the followg alteratve models have bee formulated: sales 1advert 3cet u (1) l( sales) 1l( advert) 3l( cet) u () l( sales) 1advert 3cet u (3) sales advert 3cet u (4) l( sales) l( cet) u (5) 1 31
Appedxes sales 1cet u (6) a) Usg a sample of 18 sale areas (fle advce), estmate the above models: b) I each of the followg groups select the best model, dcatg the crtera you have used. Justfy your aswer. b1) (1) ad (6) b) () ad (3) b3) (1) ad (4) b4) (), (3) ad (5) b5) (1), (4) ad (6) b6) (1), (), (3), (4), (5) ad (6) Appedx 3.1 Proof of the theorem of Gauss-Markov To prove ths theorem, the MLC assumptos 1 through 9 are used. Let us ow cosder aother estmator β whch s a fucto of y (remember that ˆ s also a fucto of y), gve by 1 β XX X A y (3-93) where A s k arbtrary matrx, that s a fucto of X ad/or other o-stochastc varables, but t s ot a fucto of y. Forβ to be ubased, certa codtos must be accomplshed. Takg (3-5) to accout, we have 1 1 β XX XA X +u AX XX XAu (3-94) Takg expectatos o both sdes of (3-94), we have 1 E( β ) AX X X XAE( u) AX (3-95) For β to be ubased, that s to say, E( β ), the followg must be accomplshed: Cosequetly, AX I (3-96) 1 β XX X A u (3-97) Takg to accout assumptos 7 ad 8, ad (3-96), the Var( β ) s equal to 3
1 1 Var( β) E(( β ( β ) E XX XAuuX XX A 1 1 1 E XX XuuX XX AA XX AA The dfferece betwee both varaces s the followg: (3-98) 1 1 ˆ Var( β ) Var( β ) XX AA XX AA (3-99) The product of a matrx by ts traspose s always a sem-postve defte matrx. Therefore, Var β Var βˆ AA ( ) ( ) 0 (3-100) The dfferece betwee the varace of a estmator β - arbtrary but lear ad ubased ad the varace of the estmator ˆβ s a sem postve defte matrx. Cosequetly, ˆβ s a Best Ubased Lear Estmator; that s to say, t s a BLUE estmator. Appedx 3. Proof: s a ubased estmator of the varace of the dsturbace I order to see whch s the most approprate estmator of, we shall frst aalyze the propertes of the sum of squared resduals. Ths oe s precsely the umerator of the resdual varace. Takg to accout (3-17) ad (3-3), we are gog to express the vector of resduals as a fucto of the regressad u 1 1 ˆ y-xβ ˆ y-x XX Xy I-X XX X y My (3-101) where M s a dempotet matrx. Alteratvely, the vector of resduals ca be expressed as a fucto of the dsturbace vector: 1 1 1 1 1 1 uˆ I-X XX X y I-X XX X X u X-XXX XXuXXX Xu X-X IX XX XuIX XX Xu Mu (3-10) Takg to accout (3-10), the sum of squared resduals (SSR) ca be expressed the followg form: uu ˆˆ ummu umu (3-103) Now, keepg md that we are lookg for a ubased estmator of, we are gog to calculate the expectato of the prevous expresso: 33
uu ˆˆ umu umu umu Etr MuutrMEuutrM I E E tre E tr trm ( k) (3-104) I dervg (3-104), we have used the property of the trace that tr( AB) = tr( BA ). Takg to accout that property of the trace, the value of trm s obtaed: 1 1 trm tr IX XX X tritrx XX X tri tri k kk Accordg to (3-104), t holds that ˆˆ E uu k Keepg (3-105) md, a ubased estmator of the varace wll be: (3-105) sce, accordg to (3-104), uu ˆˆ ˆ k (3-106) uu ˆˆ E( uu ˆˆ ) ( k) E( ˆ ) E k k k (3-107) The deomator of (3-106) s the degree of freedom correspodg to the RSS that appear the umerator. Ths result s ustfed by the fact that the ormal equatos of the hyperplae mpose k restrctos o the resduals. Therefore, the umber of degrees of freedom of the RSS s equal to the umber of observatos () mus the umber of restrctos k. Appedx 3.3 Cosstecy of the OLS estmator I appedx.8 we have proved the cosstecy of the OLS estmator ˆb the smple regresso model. Now we are gog to prove the cosstecy of the OLS vector ˆβ. Frst, the least squares estmator ˆβ, gve (3-3). may be wrtte as -1 β ˆ æ ö æ ö = β 1 + ç è X'X 1 ø çè X'u ø (3-108) Now, we take lmts the last factor of (3-108) ad call Q to the result: 1 lm X'X = Q (3-109) If X s take to be fxed repeated samples, accordg to assumpto, the (3-109) mples that Q=(l/)X'X. Accordg to assumpto 3, ad because the verse s a cotuous fucto of the orgal matrx, Q -1 exsts. Therefore, we ca wrte 34
plm( ˆ) - 1 plm é 1 β = β+ Q X'u ù êë úû The last term of (3-108) ca be wrtte as é 1 1 1 1 ùéu ù 1 úê x x x x úê u 1 úê 1 1 úê X'u = úê x x x x ú ê u 1 úê úê úê úê êëx u k1 xk xk xk úê ûë úû (3-110) éu ù 1 u 1 = [ x x x x] 1 1 = xu = xu u å = 1 êë uúû where x s the colum vector correspodg to the th observato Now, we are gog to calculate the expectato ad the varace (3-110), sce E[ uu' ] ú = 1 = 1 Eéxu ù 1 1 1 ê = E[ xu ] = x E[ u ] = X' E[ u] = 0 ë û å å var é X'X xu ù E éxu ( xu)' ù 1 1 s s ê = = X' E[ uu' ] X= = Q ë úû êë úû = s I, accordg to assumptos 7 ad 8. Takg lmts (3-11), t the follows that (3-111) (3-11) é ù s lm var êxu ë ú = lm Q= 0( Q ) =0 (3-113) û Sce the expectato of xu s detcally zero ad ts varace coverges to zero, xu coverges mea square to zero. Covergece mea square mples covergece probablty, ad so plm( xu )=0. Therefore, plm( ˆ -1-1 é1 ù -1 β) = β+ Q plm( xu ) = β + Q plm X'u = β + Q 0 = β (3-114) êë úû Cosequetly, ˆβ s a cosstet estmator. Appedx 3.4 Maxmum lkelhood estmator The method of maxmum lkelhood s wdely used ecoometrcs. Ths method proposes that the parameter estmators be those values for whch the probablty of obtag the observatos gve s maxmum. I the least squares estmato o pror assumpto was adopted. O the cotrary, the estmato by maxmum lkelhood 35
requres that statstcal assumptos about the varous elemets of the model be establshed beforehad. Thus, the estmato by maxmum lkelhood we wll adopt all the assumptos of classc lear model (CLM). Therefore, the estmato by maxmum lkelhood of β ad σ the model (3-5), we take as estmators those values that maxmze the probablty to obta the observatos a gve sample. Let us look at the procedure for obtag the maxmum lkelhood estmators β ad σ. Accordg to the CLM assumptos: N(, s ) (3-115) u 0 I The expectato ad varace of the dstrbuto of y are gve by Therefore, E( y ) = E Xβ +u =Xβ + E( u ) =Xβ (3-116) var( y ) = E yxβyxβ = Euu = I (3-117) N(, s ) (3-118) y Xβ I The probablty desty of y (or lkelhood fucto), cosderg X ad y fxed ad β ad σ varable, wll be accordace wth (3-118) equal to 1 L f( yβ, ) exp / 1 y-xβ 'y Xβ (3-119) π The maxmum for L s reached the same pot o the l(l) gve that the logarthm fucto s mootoc, ad thus, order to maxmze the fucto, we ca work wth l(l) stead of L. Therefore, l(π) l( ) 1 l( L ) (y - Xβ)'(y -Xβ) (3-10) To maxmze l(l), we dfferetate t wth respect to β ad σ : l( L) 1 ( X'y X'Xβ ) (3-11) β l( L) ( y-xβ)'(y - Xβ) (3-1) 4 Equatg (3-11) to zero, we see that the maxmum lkelhood estmator of β, deoted by β, satsfes that XX ' Xy ' (3-13) Because we assume that XX ' s vertble, β XX Xy ' (3-14) ' 1 Cosequetly, the maxmum lkelhood estmator of β, uder the assumptos of the CLM, cocdes wth OLS estmator, that s to say, β = βˆ (3-15) Therefore, 36
(y - Xβ)'(y -Xβ)=(y -Xβ)'(y ˆ -Xβˆ) uu ˆ' ˆ (3-16) Equatg (3-1) to zero ad by substtutg β by β, we obta: uu ˆ' ˆ 0 4 (3-17) where we have desgated by the maxmum lkelhood estmator of the varace of the radom dsturbaces. From (3-17), t follows that uu ˆ' ˆ (3-18) As we ca see, the maxmum lkelhood estmator s ot equal to the ubased estmator that has bee obtaed (3-106). I fact, f we take expectatos to (3-18), 1 k E Euu ˆ' ˆ (3-19) That s to say, the maxmum lkelhood estmator,, s a based estmator, although ts bas teds to zero as fty, sce k lm 1 (3-130) 37