Questions that we may have about the variables

Transcription

1 Antono Olmos, 01 Multple Regresson Problem: we want to determne the effect of Desre for control, Famly support, Number of frends, and Score on the BDI test on Perceved Support of Latno women. Dependent varable: Perceved support. Independent Varable 1: Desre for control. Measured through a questonnare. Independent Varable : Famly support. Measured through a questonnare. Independent Varable 3: Number of frends. Independent Varable 4: Score on the BDI. Questons that we may have about the varables Is the relatonshp between Perceved support (DV) and Desre for control (IV1) the same when we use a smple model than when we also nclude: Famly support (IV)? How about when we nclude Number of frends (IV3)? etc. ROADMAP3.DOC1/17/014 1

2 Antono Olmos, 01 It depends on how correlated the varables are. For most condtons, t s not. We need to translate our causal relatonshp nto a mathematcal model. Develop an equaton: Y b0 b1x1 bx b3x3 b4x4 e Y BX E or: y1 b0 1 x11 x1 x13 x14 e1 y b1 1 x1 x x3 x4 e y 3 b 1 x31 x3 x33 x 34 e 3. b y b 1 x x x x e n 4 n1 n n3 n4 n Where Ys an (n 1) vector wth the measures for the dependent varable; B s an (5 1) vector that contans the coeffcents, X s an (n 5) matrx that contans the measures of the ndependent varables (the extra column s a vector of "ones" so we can calculate the ntercept), and fnally, E s a (n 1) vector that contans the error terms: We have to fnd the parameters of the model (.e., solve the unknowns n the model, or more formally, compute the soluton). we can solve for b n our equaton: XB Y X' XB X' Y ' 1 1 X' X X' X B X' X X' Y 1 IB X X X ' Y What s the real meanng of the values we get from solvng for B? The model s takng nto account the level of redundancy among varables as t calculates the best estmates. Therefore, some books call them partal regresson coeffcents : the slopes are calculated to nclude the nfluence of other varables n the model. For example, for our model wth four IVs, the frst three coeffcents can be nterpreted as: ROADMAP3.DOC1/17/014

3 Antono Olmos, 01 b 0 estmates the mean of Y when X 1, X, X 3 and, X 4 are zero. Ths only makes sense when the ranges of both X 1 to X 4 can nclude 0. b 1 estmates the expected change n Y when we hold X, X 3 and X 4, constant. Smlarly, b explans the expected change n Y when we hold X 1, X 3, and X 4, constant. How good s the ft of the model: Estmaton of the resduals (dfference between observed and predcted scores) and the Resdual Sum of Squares (RSS): e RSS RSS y yˆ y e yˆ Estmaton of R (percentage of varance explaned): R sd y sd y y y n 1 y y n 1 y y y y Adjusted R value: takes the extra regressors nto account: k 1 Adj R R 1 R n k _ Where k = number of b s n the model. Adjusted R gves an estmate of the real change n amount of varance explaned due to addng a new regressor to the model. We can also say that adjusted R evaluates f the mprovement n the model s small relatve to the ncrease n complexty. Multple correlaton coeffcent (measures the assocaton between the DV and an optmal combnaton of the IV s): mult_ corr r YY, ROADMAP3.DOC1/17/014 3

4 Antono Olmos, 01 Test of sgnfcance: Omnbus test (checks f at least one of the slopes s sgnfcant): F MS MS REGRESSION ERROR Yˆ Y p 1 y Yˆ n p wth (k-1, n-k) degrees of freedom. The standard error of the coeffcents s one of the by-products of the matrx approach: se _ B sdev X ' X 1 resd 1 Where X' X, represents the correspondng element of the man dagonal of the nverse matrx of crossproducts, and (sdev (resd) ) s the standard devaton of the resduals rased to the square (same as the Mean of Squares of Error). The t-test: t_ b b se_ B wth (n-k) degrees of freedom. Confdence Intervals for all the b s: CI_b (1-) = b ± (t tables ) (se_b ) Confdence Intervals mean of a predcted value (answers the queston: What s the Confdence Interval for the mean (Y) when (X 1, X, X 3, X 4 ) are ): The trcky part s fgurng out the standard error, because we have several IV s. Ask me (or check: Montgomery, D. C., & Peck, E. A. (198). Introducton to lnear regresson analyss. NY: John Wley., pages 17-18). CI_ Y (1-) = Y ± (t tables ) (se_ Y ) What f the Independent Varables are correlated? MULTICOLLINEARITY: any or all of the IVs are lnearly related wth any or all of the others. Sources of Multcollnearty: ROADMAP3.DOC1/17/014 4

5 Antono Olmos, 01 Data collecton method: when we sample only a lmted regon of the populaton. By dong so, we may end up wth strongly-correlated varables. Constrants on the model: In ths case, t does not matter how we sample, we wll always get that constrant Choce of the model: models that use polynomal terms (lke age ), n addton to the lnear term (.e., age). Over defned model: a model wth more IV than cases. Very common n psychology and health scences (e.g., clncal cases). What f we have multcollnearty? If we have multcollnearty, we may have a msleadng nterpretaton of the regresson coeffcents (coeffcents cannot be trusted). The prncpal problem wth ths estmates s the extrapolaton to other samples/other values beyond those used to estmate the coeffcents. The coeffcents are unrelable because they wll change from sample to sample. If we have multcollnearty, the standard error of the coeffcents wll be huge. Thus, slght dfferent samples wll gve very dfferent estmates of the same coeffcent. Theoretcally (.e., after an nfnte number of samples are taken), the value of the coeffcents wll converge to the mean. However, n any gven sample, the value may be way off Even, of opposte sgn! Because of the huge standard error, and naccurate estmaton of the coeffcents, we loose power (.e., t s harder to reject the null hypothess that the b s are dfferent from zero). How can we spot Multcollnearty? Check the correlaton matrx. If we fnd large correlatons among ndependent varables, then we know that we have the problem. ROADMAP3.DOC1/17/014 5

6 Antono Olmos, 01 Check the determnant of the (X'X) matrx. The values of the man dagonals of the Inverse of the Correlaton matrx among IV s (C T C) -1 matrx are equal to: T 1 1 C C, 1 R ( b) Where R ( b) s the coeffcent of determnaton we get when X was regressed on the remanng p-1 regressors. The elements of the man dagonal are the so-called Varance Inflaton Factor (VIF) (reported by SPSS). The value: ( ) 1 R b Is called tolerance. We can see that we want ths value to be close to 1, because that means that R ( b) s almost zero. Ths value s also reported n SPSS. Check the value of the standard error of the coeffcents (compare t to se( b ) when t s the only ndependent varable n the model). Compare the sgnfcance values of both the F and the t's. Multcollnearty sometmes makes the F-test to be sgnfcant, whle the t's are not (because the standard errors of the coeffcents are huge). The sgns and magntudes of the regresson coeffcents can also sometmes provde an ndcaton of multcollnearty. If addng or removng an IV produces wld changes n the estmates, then there s multcollnearty. In addton, f deleton of one or more data ponts produces wld varatons n the coeffcents, that may be an ndcaton of multcollnearty. If the values of the standardzed regresson coeffcents are larger than ether +1 or -1, t means that we have problems of multcollnearty. If the sgns of the coeffcents are contrary to what you know/expect, then be alert about the possblty of multcollnearty. ROADMAP3.DOC1/17/014 6

7 Antono Olmos, 01 If the condton ndex for one of the egenvalues s too hgh, then we may have a problem wth multcollnearty. If a hgh proporton of the varance of two or more coeffcents s assocated wth the same egenvalue, then that s a clear ndex of multcollnearty. What to do f we have multcollnearty? Get rd of some of the varables that are creatng the problems. Try to combne ther scores nto one sngle value Keep the model, under the understandng that generalzatons beyond the sample are rsky. Do not try to nterpret the b s. OK as long as multcollnearty s an ntegral part of the model (the populaton always shows the same level of relaton among ndependent varables). CI for predcton are not affected. However, do not try to predct outsde the ranges of your varables n the model. Predcton s better f close to the means of the varables. If polynomal models, try centerng them, or use orthogonal polynomals. Use herarchcal models. Try to fnd-out what s the latent construct behnd the correlated varables (do factor analyss). Try the technque called Rdge Regresson, whch s more robust to multcollnearty. Add more cases. Ths of course n case you suspect that the multcollnearty problem s due to samplng bas (check causes for multcollnearty). ROADMAP3.DOC1/17/014 7