Week 3: Resdual Analyss (Chapter 3) Propertes of Resduals Here s some propertes of resduals, some of whch you learned n prevous lectures:. Defnton of Resdual: e = Y Ŷ. ε = Y E( Y ) 3. ε ~ NID( 0 σ ), 4. mean: e = 0 e = 0 5. varance: ( e e) n = e n = SSE = MSE n Semstudentzed Resduals We wll use standardzed resduals n some of our analyses of resduals. The standardzaton formula, e e e e =, s often used to standardze resduals. KNNL on page 03 MSE MSE * = explan ths standardzaton would create a studentzed resdual f MSE were an estmate of the standard devaton of the resdual. However, ths formula does not produce truly studentzed resduals because MSE s only an approxmaton of the standard devaton of e. They wll dscuss how to calculate studentzed resduals n chapter 0. Stll, ths formula s the bass of many resdual analyss technques, and t s the formula used by many statstcal software packages, such as SAS, to standardze the resduals.
Sx Areas n Whch We Wll Use Resdual Analyss. regresson not lnear. non-constant varance 3. ndependence of resduals 4. outlers 5. normalty of errors 6. mportant predctor (ndependent) varables. Vsual Dagnostcs for Resduals Resdual plots are qute useful for examnng resduals n the above sx categores (Fgure 3.4, page 06). You can also use a normal probablty plot to examne f the resduals are normally dstrbuted (normal probablty plots are often standard output n statstcal analyss software). Statstcal Tests for Resduals Normalty: You can use the Shapro-Wlks statstc (ths s standard output n the Proc Freq procedure n SAS). KNNL also menton the Correlaton test for Normalty, whch they say s easer to use than the Shapro-Wlks test. You can also use standard goodness of ft tests, lke the ch-square or Kolmogorov-Smrnov tests. Autocorrelaton (randomness): You can use the Durbn-Watson statstc to determne f you have sgnfcant autocorrelaton (ths s standard output n SAS). You can also use a Runs test.
Non-constant varance (heteroscedastcty): KNNL present two tests that can be used to check for constant varance: ) the Modfed Levene or Brown-Forsythe test, and ) the Breusch-Pagan test. The Modfed Levene test does not requre that the errors be normally dstrbuted, unlke the Breusch-Pagan test. KNNL report that the Modfed Levene test s actually qute robust aganst severe departures from normalty. The sample sze, though, does need to be large. KNNL present an example of the Modfed Levene test on page 7 and an example of the Breusch-Pagan test on page 9. Lack of Ft Test A Lack of Ft test s used to determne f a regresson functon adequately fts the data. Ths test assumes that the Y s are ndependent, normally dstrbuted, and have constant varance. It also requres that replcates at one or more levels of X are avalable. When replcates are avalable, the error can be dvded nto two components: ) pure error, and ) lack of ft error. The pure error component recognzes that replcatons exst for some levels of X. The sums of squares for the pure error can be expressed as: c n ( Y Y ) SSPE =, = = where c = number of levels of X and = number of observatons for a gven level of X. The degrees of freedom assocated wth SSPE = n c. Wth ths nformaton, we can compute an unbased estmator of the error varance: 3
SSPE MSPE =. n c The lack of ft component s smply the dfference between the overall error, SSE, and the pure error component, SSPE: SSLF = SSE SSPE, or Y Ŷ = Y Y + Y Ŷ. SSLF can be expressed drectly as: SSLF = c = n ( Y Ŷ ), wth c degrees of freedom. The lack of ft aspect can be seen n the dfference, Y Ŷ. If the dfference s small, then you can conclude that the regresson model s a better ft than f the dfference s large. Wth SSLF and ts degrees of freedom, we can calculate the mean square for the lack of ft: SSLF MSLF =. c Now that we have expressons for two mean squares, we can construct an F-statstc for our lack of ft test: MSLF F =. MSPE We can carry out the test as an ANOVA n the usual fashon. Note: KNNL present the lack of ft test n the context of the general lnear test on page 7. Begnnng on page, they present the Full Model and the Reduced Model, whch provde the approprate sums of squares to construct the F-statstc for the lack of ft test. 4
Example: Problem 3.5 on page 50. In ths example, a chemst measured the concentraton of a soluton (Y) over tme (X) (n = 5 solutons). The n = 5 solutons were randomly dvded nto fve sets of three, and the fve sets were measured after, 3, 5, 7, and 9 hours, respectvely. Here are the data: Hypotheses Ho : E Ha : E α = 0.05 ( Y ) = β0 + βx ( Y ) β0 + βx X Y 9 0.07 9 0.09 3 9 0.08 4 7 0.6 5 7 0.7 6 7 0. 7 5 0.49 8 5 0.58 9 5 0.53 0 3. 3.5 3.07 3.84 4.57 5 3.0 Decson Rule If F F -α, c, n c = F.95, 3, 0 = 3.7, then reect Ho and conclude that the regresson functon does not adequate ft the data (.e., a sgnfcant lack of ft exsts). Results Regresson Equaton: Y =.5753 0.34 * X. Snce F = 58 >> 3.7 (p << 0.000), reect Ho and conclude there s a lack of ft. 5
ANOVA table Source df SS MS F P - value Regresson.597.597 Error n = 3.946 0.5 Lack of Ft c = 3.767 0.94 58.75 < 0.000 Pure Error n c = 0 0.574 0.057 Total n = 4 5.58 Remedal Measures If the SLR model does not ft the data, then you have two choces:. Fnd a new model form;.e., pck a nonlnear model.. Transform the data. Data Transformatons The followng transformatons on X wll often lnearze a nonlnear relatonshp. If they do not adequately work, then you should use a nonlnear regresson model, whch we wll dscuss later n the class. X ' = X X ' = X X ' = log X or ln X ' X = X 6
The followng transformatons of Y (smlar to the ones above for X) wll often stablze the varance and/or fx non-normalty n the errors. Often, these transformatons on Y wll fx nonnormalty and stablze the varance smultaneously. Y ' = Y Y ~ Posson Y ' = arcsn Y Y ~ Bnomal Y ' = Y Y ' = log Y or ln Y Y ~ Lognormal. You can also use weghted regresson to stablze the varance. We wll dscuss ths topc later n Chapter. Also, KNNL menton Box-Cox transformatons (page 34) as remedal measures for unequal varances, nonnormalty, etc. Box-Cox transformatons are useful when you don t know exactly whch transformaton on Y to use. The Box-Cox procedure utlzed Maxmum Lkelhood estmaton to dentfy the approprate transformaton on Y from a famly of power functons. They also present an approxmaton technque that does not requre you to mnmze the lkelhood functon snce many statstcal analyss packages typcally do not allow you to use Box-Cox transformatons (see page 36). 7