The leverage statistic, h, also called the hat-value, is available to identify cases which influence the regression model more than others.

Transcription

1 Outliers Outliers are data points which lie outside the general linear pattern of which the midline is the regression line. A rule of thumb is that outliers are points whose standardized residual is greater than 3.3 (corresponding to the.001 alpha level). The removal of outliers from the data set under analysis can at times dramatically affect the performance of a regression model. Outliers should be removed if there is reason to believe that other variables not in the model explain why the outlier cases are unusual -- that is, these cases need a separate model. Alternatively, outliers may suggest that additional explanatory variables need to be brought into the model (that is, the model needs respecification). Another alternative is to use robust regression, whose algorithm gives less weight to outliers but does not discard them. The leverage statistic, h, also called the hat-value, is available to identify cases which influence the regression model more than others. Belsley, Kuh, and Welsch (1980) define the leverage (h i ) of the ith observation as h i = 1 n % (x i & x)2 (n&1)s 2 x Leverage assesses how far away a value of the independent variable value is from the mean value: the farther away the observation the more leverage it has. From the definition you can see that leverage is mitigated by a larger sample size (any single point should have less influence) and by a larger variance of the independent variable (again, any single point should have less influence). 0<h<1 The leverage statistic varies from 0 (no influence on the model) to 1 (completely determines the model). A rule of thumb is that cases with leverage under.2 are not a problem, but if a case has leverage over.5, the case has undue leverage and should be examined for the possibility of measurement error or the need to model such cases separately. STATA command predict h, hat. Cook's distance, D, is another measure of the influence of a case. Cook's distance measures the effect of deleting a given observation. Observations with larger D values than the rest of the data are those which have unusual leverage. D > 4/n the criterion to indicate a possible problem. STATA command predict D, cooksd

2 dfbetas, is another statistic for assessing the influence of a case. If dfbetas > 0, the case increases the slope; if <0, the case decreases the slope. The case may be considered an influential outlier if dfbetas > 2/%n. STATA command dfbeta creates dfbeta s for all variables. or predict DFx1, dfbeta(x1) for individual variables dffit. DfFit measues how much the estimate changes as a result of a particular observation being dropped from analysis. dfits is defined as h dfits i = Rstudent i. Where Rstudent is the studentized residual. 1&h i The case may be considered an influential outlier if DFITS> 2/%k/n STATA command predict DFITS, dfits Studentized residuals and deleted studentized residuals are also used to detect outliers with high leverage. A "studentized residual" is the observed residual divided by the standard deviation. The "studentized deleted residual," also called the "jacknife residual," is the observed residual divided by the standard deviation computed with the given observation left out of the analysis. Analysis of outliers usually focuses on deleted residuals. Other synonyms include externally studentized residual or, misleadingly, standardized residual. There will be a t value for each residual, with df - n - k - 1, where k is the number of independent variables. When t exceeds the critical value for a given alpha level (ex.,.05) then the case is considered an outlier. In a plot of deleted studentized residuals versus ordinary residuals, one may draw lines at plus and minus two standard units to highlight cases outside the range where 95% of the cases normally lie; points substantially off the straight line are potential leverage problems. STATA command predict student, rstudent or predict standard, rstandard Partial regression plots, also called partial regression leverage plots or added variable plots, are yet another way of detecting influential sets of cases. Partial regression plots are a series of bivariate regression plots of the dependent variable with each of the independent variables in turn. The plots show cases by number or label instead of dots. One looks for cases which are outliers on all or many of the plots.

3 STATA command avplots Example using the Murder.dta data set. reg mrdrte exec unem d90 d93 Source SS df Number of obs= 153 F( 4, 148) = 3.05 Model Prob > F = Residual R-squared = Adj R-squared= Total Root E = mrdrte Coef. Std. Err. t P>t [95% Conf. Interval] exec unem d d _cons predict e, residual predict yhat predict standard, rstandard predict student, rstudent predict h, hat predict D, cooksd predict DFITS, dfits predict W, welsch dfbeta DFexec: DFbeta(exec) DFunem: DFbeta(unem) DFd90: DFbeta(d90) DFd93: DFbeta(d93)

4 rvplot Residuals Fitted values To create a standardized residual plot graph twoway scatter standard yhat, yline(0)

5 Standardized residuals Fitted values To identify the outliers. graph twoway scatter standard yhat, yline(0) mlabel(state) Standardized residuals NH MD CAFL GA DE CT NC MD GA MD HI VA SCNCNCSC CA GA AZ AR MI MA NE MO HI KS AZ AZAL AL CO AK FL AL HI KS NV NVNV IL IL MI MOIL RJ VA TN TN TN VT ME AR AR DE COCTCT DE CO AK FL IAIA IA KS KYKYMO MI NE NE PA OR NM NM MN MN NJ OHPA OKOK SC SD SD UT NM ND OH ND SD WI VA WI WI PA MN KY ND UT UT VTWA WY WY VT ME IDMTMT OR WA NHNH IDID MAOR NJ RIWA WY ME CA AK Fitted values

6 avplots, mlabel(state) e( mrdrte X ) AK AK NM ME RI WY NJ OR PA MD AR CO IDNH OK NV IDMT MI AK RI VT CO NM MO IL OR MA KY OH CA SC OR IAND PA MN HI AZ TN MD IADE CT ME NJ VT WA CO KS NE MN WI KS ME CA NC NJ RI NE SD VT WY CT NV MA DE HI NH SC NE AL NC UT OK AR NC SC AZ GANV VA VA AL MO MO FL FL FL GA VA e( mrdrte X ) NH NENE DE MA CT SDVT HI RI HI UT VA MD NJ IASD HI ND VA ME NC FL GA NC MD SD WI ND NE VA KS IAUT MN ND CO KS WI SC NV GA DE WY MN VT IAMN PA TN CA WA CO NJ VT OK ME DE CT MO TN AR GA MT WI AZ AZ MD OR KY CT NV PA IDWY MO FL MO CA UT OR OK OH FL TN NH MT KY ID MA OH PA MI IL NM AL OK IL OR MT NJ WA SC NM WA AK AR AL CO AK MI RI ME ID AR MI WY KY NMCA AK e( exec X ) coef = , se = , t = e( unem X ) coef = , se = , t = 3.08 e( mrdrte X ) NHMA DE CT VT HI RI NJ VA MD SDME NC FL NE KS ND MN GA IA SC CA PA WI AZ OR NV MO UT TN OH MT ILOK WA CO AL ID AR MI KY NM AK NE NC SDUT WY IAHI ND WI VA KS MN CO DE WY VT TN GA OK AZ AR MT ID KY NV CT MD MO OH NH MA FL PA MI IL OR NJ SC WA NM AK CA NEHI NC VA RI ME SDND IA UT KS WI MD SC NV MN WA CO NJ VT ME CT DE TN AZ GA PA WY OR OK CA MO NH OH KY ID MT FL ILNM AL AR MA RI AK MI e( d90 X ) coef = , se = , t = 1.47 e( mrdrte X ) GA FLVA SC NC AL NV UT NH DE MA CT MD VT HI RI NJ SD ME KS NE ND MN IA PA CA WI OR AZ MO TN OH IL OK MI MT WA CO AR ID KY NM AK WY FLMO AL VA NV GA SC AR UT OK NC MD NE HI TN CA SD ND IAWI KS MN WA CO NJ VT CT ME DE AZ WY PA OR OH KY ILNM NH IDMT MA RI AK MI VA FLMO GA NC AZ OK AR AL CA UT WY SC NE SD WA IAHI ND WI KS MN CO TN NV MD DE VT KY MT ID CT OH NH MA PA MI IL OR NJ NM AK RI ME e( d93 X ) coef = , se = , t =.91 To graphically measure the influence of observations graph twoway scatter standard yhat [aweight=d], msymbol(oh) yline(0) Standardized residuals Fitted values Leverage versus residual squared plot marks the means of leverage and squared residuals. Leverage

7 tells us how much potential for influencing the regression an observation has. lvr2plot To examine the numerical measure for outliers. leverage h>2k/n Cook s D>4/n DFITS>2/%k/n Welsch s W>3/%k DFBETA>2/%n For example, sort D list state yhat D DFITS W in -5/l state yhat D DFITS W

8 Recall that D > 4/n indicates a possible problem. With n=153, D > DFITS> 2/%k/n may be considered an influential outlier. With n=153, k=4, DFITS>