Standard errors for regression coefficients; Multicollinearity

Transcription

1 Standard rror for rgrion cofficint; Multicollinarit Standard rror. Rcall that b k i a point timat of β k. Bcau of ampling variabilit, thi timat ma b too high or too low. bk, th tandard rror of b k, giv u an indication of how much th point timat i likl to var from th corrponding population paramtr. W will now broadn our arlir dicuion. Lt H th t of all th X (indpndnt) variabl. Lt G k th t of all th X variabl xcpt. Th following formula thn hold: Gnral ca: bk Gk ) ( N 1) Th firt formula u th tandard rror of th timat. Gk R YH )( N K 1) Th cond formula mak it clarr how tandard rror ar rlatd to R. IV ca bk ( R ) ( N 1) 1 Whn thr ar onl IV, R XkGk R 1. R Y1 ( R )( N K 1) 1 1 IV ca b X ( N 1) Whn thr i onl 1 IV, R XkGk 0. R ( N K 1) Y X For xampl, if K 5, thn R YH 5 i th multipl R5 obtaind b rgrion Y on X 1, X, X 3, X 4, and X 5 ; and, if w wantd to know b3 (i.. th tandard rror for X 3 ) thn R X3G3 5 would b th multipl R5 obtaind b rgring X 3 on X 1, X, X 4, and X 5. Not that, whn thr ar indpndnt variabl, R G1 5 R XG 5 R 1 5. Qution. Suppo K 1, i.. thr i onl 1 indpndnt variabl. What i th corrct formula thn? Anwr. Whn K 1, R XkGk 5 0 (bcau thr ar no othr X to rgr on X 1 ). Th gnral formula thn bcom th am a th formula w prntd whn dicuing bivariat rgrion. Qution. What happn a R XkGk 5 gt biggr and biggr? Standard rror for rgrion cofficint; Multicollinarit - Pag 1

2 Anwr. A R XkGk 5 gt biggr and biggr, th dnominator in th abov quation gt mallr and mallr, hnc th tandard rror gt largr and largr. For xampl: If R 1 5 0, and nothing l chang, thn, b1 1 ) ( N 1) (1 0)0.05(19) R Y1 )( N K 1) (1 0)(17) b 1 ) X ( N 1) (1 0)9.8(19) R Y1 )( N K 1) 1 X (1 0)(17) If R and nothing l chang, thn, b1 1 ) ( N 1) (1.5)0.05(19) R Y1 )( N K 1) (1.5)(17) b 1 ) X ( N 1) (1.5)9.8(19) R Y1 )( N K 1) 1 X (1.5)(17) Similarl, ou can how that, if R 1 5.5, thn b1.95 and b.4. Qution. Suppo R XkGk 5 i vr larg, a, clo to 1. What happn thn? Anwr. If R XkGk 5 1, thn 1 - R XkGk 5 0, which man that th tandard rror bcom infinitl larg. Ergo, th clor R XkGk 5 i to 1, th biggr th tandard rror gt. Put anothr wa, th mor corrlatd th X variabl ar with ach othr, th biggr th tandard rror Standard rror for rgrion cofficint; Multicollinarit - Pag

3 bcom, and th l likl it i that a cofficint will b tatiticall ignificant. Thi i known a th problm of multicollinarit. Intuitivl, th raon thi problm occur i a follow: Th mor highl corrlatd indpndnt variabl ar, th mor difficult it i to dtrmin how much variation in Y ach X i rponibl for. For xampl, if X 1 and X ar highl corrlatd (which man th ar vr imilar to ach othr) it i difficult to dtrmin whthr X 1 i rponibl for variation in Y, or whthr X i. A a rult, th tandard rror for both variabl bcom vr larg. In our currnt xampl, if R , thn b1.933 and b.765. Not that, undr th condition, nithr cofficint would b ignificant at th.05 lvl, vn though thir combind ffct ar tatiticall ignificant. Commnt: 1. It i poibl for all indpndnt variabl to hav rlativl mall mutual corrlation and t to hav om multicollinarit among thr or mor of thm. Th multipl corrlation R XkGk can indicat thi.. Whn multicollinarit occur, th lat-quar timat ar till unbiad and fficint. Th problm i that th timatd tandard rror of th cofficint tnd to b inflatd. That i, th tandard rror tnd to b largr than it would b in th abnc of multicollinarit bcau th timat ar vr nitiv to chang in th ampl obrvation or in th modl pcification. In othr word, including or xcluding a particular variabl or crtain obrvation ma gratl chang th timatd cofficint. 3. On wa multicollinarit can occur i if ou accidntall includ th am variabl twic,.g. hight in inch and hight in ft. Anothr common rror occur whn on of th X i computd from th othr X (.g. Famil incom Wif incom + Huband incom), and th computd variabl and th variabl ud to comput it ar all includd in th rgrion quation. Impropr u of dumm variabl (which w will dicu latr) can alo lad to prfct collinarit. Th rror ar all avoidabl. Howvr, othr tim, it jut happn to b th ca that th X variabl ar naturall highl corrlatd with ach othr. 4. Man computr program for multipl rgrion hlp guard againt multicollinarit b rporting a tolranc figur for ach of th variabl ntring into a rgrion quation. Thi tolranc i impl th proportion of th varianc for th variabl in qution that i not du to othr X variabl; that i, Tolranc 1 - R XkGk. A tolranc valu clo to 1 man ou ar vr af, whra a valu clo to 0 how that ou run th rik of multicollinarit, and poibl no olution, b including thi variabl. Not, incidntall, that th tolranc appar in th dnominator of th formula for th tandard rror. A th tolranc gt mallr and mallr (i.. a multicollinarit incra) tandard rror gt biggr and biggr. Alo uful i th Varianc Inflation Factor (VIF), which i th rciprocal of th tolranc. Thi how u how much th varianc ar inflatd b multicollinarit,.g. if th VIF i 1.44, multicollinarit i cauing th varianc of th timat to b 1.44 tim largr than it would b if th indpndnt variabl wr uncorrlatd (maning that th tandard rror i tim largr). Standard rror for rgrion cofficint; Multicollinarit - Pag 3

4 5. Thr i no impl man for daling with multicollinarit (othr than to avoid th ort of common mitak mntiond abov.) Som poibiliti: a. Exclud on of th X variabl - although thi might lad to pcification rror b. Find anothr indicator of th concpt ou ar intrtd in, which i not collinar with th othr X. c. Put contraint on th ffct of variabl,.g. rquir that or mor variabl hav qual ffct (or ffct of qual magnitud but oppoit dirction.) For xampl, if ar of ducation and ar of job xprinc wr highl corrlatd, ou might comput a nw variabl which wa qual to EDUC + JOBEXP, and u it intad. d. Collct a largr ampl, inc largr ampl iz rduc th problm of multicollinarit b rducing tandard rror.. In gnral, b awar of th poibl occurrnc of multicollinarit, and know how it might ditort our paramtr timat and ignificanc tt. Hr again i an xpandd printout from SPSS that how th tolranc and VIF: Modl 1 (Contant) EDUC JOBEXP Untandardizd Cofficint a. Dpndnt Variabl: INCOME Standardizd Cofficint Cofficint a 95% Confidnc Intrval for B t Sig. Lowr Bound Uppr Bound Corrlation Zro-ordr Partial Part Collinarit Statitic VIF B Std. Error Bta Std. Error Tolranc Standard rror for rgrion cofficint; Multicollinarit - Pag 4

5 Anothr xampl. Lt tak anothr look at on of our homwork problm. W will xamin th tolranc and how how th ar rlatd to th tandard rror. Man Std Dv Varianc Labl XHWORK TIME ON HOMEWORK PER WEEK XBBSESRW SES COMPOSITE SCALE SCORE ZHWORK TIME ON HOMEWORK PER WEEK XTRKACAD X IN ACADEMIC TRACK N of Ca 9303 Equation Numbr 1 Dpndnt Variabl.. XHWORK TIME ON HOMEWORK PER WEEK Multipl R.4098 R Squar Standard Error Anali of Varianc DF Sum of Squar Man Squar Rgrion Ridual F Signif F Variabl in th Equation Variabl B SE B 95% Confdnc Intrvl B Bta SE Bta Corrl Part Cor XBBSESRW ZHWORK XTRKACAD (Contant) Variabl in th Equation Variabl Partial Tolranc VIF T Sig T XBBSESRW ZHWORK XTRKACAD (Contant) To implif th notation, lt Y XHWORK, X 1 XBBSESRW, X ZHWORK, X 3 XTRKACAD. Th printout tll u N 9303, SSE ,.65806, R Y ,.913, 1.686,.930, 3.467, Tolranc X > R G Tolranc X > R XG Tolranc X > R X3G Th high tolranc and th big ampl iz trongl uggt that w nd not b worrid about multicollinarit in thi problm. W will now comput th tandard rror, uing th information about th tolranc. Standard rror for rgrion cofficint; Multicollinarit - Pag 5

6 b1 (1- R X 1G1 1 - RYH ) (N - K - 1) x b X G 1 RYH ) (N K 1) x b3 1 RYH ) (N K 1) X 3G3 x3 b3 x A ou can, our computd tandard rror ar conitnt with SPSS. W will hav a mor xtniv dicuion of multicollinarit in Stat II. Standard rror for rgrion cofficint; Multicollinarit - Pag 6