Chapter 14 Simple Linear Regression

Transcription

1 Sldes Prepared JOHN S. LOUCKS St. Edward s Unverst Slde Chapter 4 Smple Lnear Regresson Smple Lnear Regresson Model Least Squares Method Coeffcent of Determnaton Model Assumptons Testng for Sgnfcance Usng the Estmated Regresson Equaton for Estmaton and Predcton Computer Soluton Resdual Analss: Valdatng Model Assumptons Slde Smple Lnear Regresson Model The equaton that descres how s related to and an error term s called the regresson model. The smple lnear regresson model s: = β + β +ε β and β are called parameters of the model, ε s a random varale called the error term. Slde 3

2 Smple Lnear Regresson Equaton The smple lnear regresson equaton s: E() = β + β Graph of the regresson equaton s a straght lne. β s the ntercept of the regresson lne. β s the slope of the regresson lne. E() s the epected value of for a gven value. Slde 4 Smple Lnear Regresson Equaton Postve Lnear Relatonshp E() Regresson lne Intercept β Slope β s postve Slde 5 Smple Lnear Regresson Equaton Negatve Lnear Relatonshp E() Intercept β Regresson lne Slope β s negatve Slde 6

3 Smple Lnear Regresson Equaton No Relatonshp E() Intercept β Regresson lne Slope β s Slde 7 Estmated Smple Lnear Regresson Equaton The estmated smple lnear regresson equaton ŷ = + The graph s called the estmated regresson lne. s the ntercept of the lne. s the slope of the lne. ŷŷ s the estmated value of for a gven value. Slde 8 Estmaton Process Regresson Model = β + β +ε Regresson Equaton E() = β + β Unknown Parameters β, β Sample Data:.... n n and provde estmates of β and β Estmated Regresson Equaton ŷ = + Sample Statstcs, Slde 9 3

4 Least Squares Method Least Squares Crteron mn ( $ ) = oserved value of the dependent varale for the th oservaton ^ = estmated value of the dependent varale for the th oservaton Slde Least Squares Method Slope for the Estmated Regresson Equaton = ( )( ) = ( ) Slde Least Squares Method -Intercept for the Estmated Regresson Equaton = = value of ndependent varale for th oservaton = value of dependent varale for th _ oservaton = mean value for ndependent varale _ = mean value for dependent varale n = total numer of oservatons Slde 4

5 Smple Lnear Regresson Eample: Reed Auto Sales Reed Auto perodcall has a specal week-long sale. As part of the advertsng campagn Reed runs one or more televson commercals durng the weekend precedng the sale. Data from a sample of 5 prevous sales are shown on the net slde. Slde 3 Smple Lnear Regresson Eample: Reed Auto Sales Numer of TV Ads 3 3 Numer of Cars Sold Slde 4 Smple Lnear Regresson (Contnued) Reed Auto Sales X= # of TV ads Y= # of Cars sold ( ) ( ) ( ) ( ) ( )( ) =, = = 4 = 4 = ( ) = 4 ( )( ) = Slde 5 5

6 Estmated Regresson Equaton Slope for the Estmated Regresson Equaton ( )( ) = = = 5 ( ) 4 -Intercept for the Estmated Regresson Equaton = = 5() = Estmated Regresson Equaton ˆ = + 5 Slde 6 Scatter Dagram and Trend Lne 3 5 Cars Sold 5 5 = TV Ads Slde 7 Coeffcent of Determnaton Relatonshp Among SST, SSR, SSE SST = SSR + SSE ( ) = ( ˆ ) + ( ˆ ) SST = total sum of squares SSR = sum of squares due to regresson SSE = sum of squares due to error Slde 8 6

7 Coeffcent of Determnaton The coeffcent of determnaton s: r = SSR/SST SSR = sum of squares due to regresson SST = total sum of squares Slde 9 Coeffcent of Determnaton r = SSR/SST = /4 =.877 The regresson relatonshp s ver strong; 88% of the varalt n the numer of cars sold can e eplaned the lnear relatonshp etween the numer of TV ads and the numer of cars sold. Slde Sample Correlaton Coeffcent r = (sgn of ) r = (sgn of ) Coeffcent of Determnatonon r = the slope of the estmated regresson equaton ˆ = + Slde 7

8 Sample Correlaton Coeffcent r = (sgn of ) r The sgn of n the equaton ˆ = + 5 s +. r r = r = Slde Assumptons Aout the Error Term ε. The error ε s a random varale wth mean of zero.. The varance of ε, denoted σ, s the same for all values of the ndependent varale. 3. The values of ε are ndependent. 4. The error ε s a normall dstruted random varale. Slde 3 Testng for Sgnfcance To test for a sgnfcant regresson relatonshp, we must conduct a hpothess test to determne whether the value of β s zero. Two tests are commonl used: t Test and F Test Both the t test and F test requre an estmate of σ, the varance of ε n the regresson model. Slde 4 8

9 Testng for Sgnfcance An Estmate of σ The mean square error (MSE) provdes the estmate of σ, and the notaton s s also used. SSE = s = MSE = SSE/(n ) ( ˆ ) = ( ) Slde 5 Testng for Sgnfcance An Estmate of σ To estmate σ we take the square root of σ. The resultng s s called the standard error of the estmate. s = MSE = SSE n Slde 6 Testng for Sgnfcance: t Test Hpotheses Test Statstc H : β = a: β H t = s Slde 7 9

10 Testng for Sgnfcance: t Test Rejecton Rule Reject H f p-value < α or t < -t α/ or t > t α/ t α/ s ased on a t dstruton wth n - degrees of freedom Slde 8 Testng for Sgnfcance: t Test. Determne the hpotheses. H : β = a: β H. Specf the level of sgnfcance. α =.5 3. Select the test statstc. t = s 4. State the rejecton rule. Reject H f p-value <.5 or t > 3.8 (wth 3 degrees of freedom) Slde 9 Testng for Sgnfcance: t Test 5. Compute the value of the test statstc. 5 t = = = 4.63 s.8 6. Determne whether to reject H. t = 4.54 provdes an area of. n the upper tal. Hence, the p-value s less than.. (Also, t = 4.63 > 3.8.) We can reject H. Slde 3

11 Confdence Interval for β We can use a 95% confdence nterval for β to test the hpotheses just used n the t test. H s rejected f the hpotheszed value of β s not ncluded n the confdence nterval for β. Slde 3 Confdence Interval for β The form of a confdence nterval for β s: s the pont estmator where t α / ± t s ± α / s the t value provdng an area of α/ n the upper tal of a t dstruton wth n - degrees of freedom t α /s s the margn of error Slde 3 Confdence Interval for β Rejecton Rule Reject H f s not ncluded n the confdence nterval for β. 95% Confdence Interval for β ± t s = 5 +/- 3.8(.8) = 5 +/ ± tα / s or.56 to 8.44 Concluson s not ncluded n the confdence nterval. Reject H Slde 33

12 Hpotheses Testng for Sgnfcance: F Test Test Statstc H : β = a: β H F = MSR/MSE Slde 34 Rejecton Rule Testng for Sgnfcance: F Test Reject H f p-value < α or F > F α F α s ased on an F dstruton wth degree of freedom n the numerator and n - degrees of freedom n the denomnator Slde 35 Testng for Sgnfcance: F Test. Determne the hpotheses. H : β = H : a β. Specf the level of sgnfcance. α =.5 3. Select the test statstc. F = MSR/MSE 4. State the rejecton rule. Reject H f p-value <.5 or F >.3 (wth d.f. n numerator and 3 d.f. n denomnator) Slde 36

13 Testng for Sgnfcance: F Test 5. Compute the value of the test statstc. F = MSR/MSE = /4.667 = Determne whether to reject H. F = 7.44 provdes an area of.5 n the upper tal. Thus, the p-value correspondng to F =.43 s less than (.5) =.5. Hence, we reject H. The statstcal evdence s suffcent to conclude that we have a sgnfcant relatonshp etween the numer of TV ads ared and the numer of cars sold. Slde 37 Some Cautons aout the Interpretaton of Sgnfcance Tests Rejectng H : β = and concludng that the relatonshp etween and s sgnfcant does not enale us to conclude that a cause-and-effect relatonshp s present etween and. Just ecause we are ale to reject H : β = and demonstrate statstcal sgnfcance does not enale us to conclude that there s a lnear relatonshp etween and. Slde 38 Usng the Estmated Regresson Equaton for Estmaton and Predcton Confdence Interval Estmate of E( p ) $ p ± t α / s $ p Predcton Interval Estmate of p ± t s p ± α / nd confdence coeffcent s - α and t α/ s ased on a t dstruton wth n - degrees of freedom Slde 39 3

14 Pont Estmaton If 3 TV ads are run pror to a sale, we epect the mean numer of cars sold to e: ^ = + 5(3) = 5 cars Slde 4 Confdence Interval for E( p ) Ecel s Confdence Interval Output D E F G CONFIDENCE INTERVAL p 3 3 ar. 4 p - ar. 5 ( p - ar). 6 Σ ( p - ar) 4. 7 Varance of hat. 8 Std. Dev of hat t Value 3.84 Margn of Error 4.68 Pont Estmate 5. Lower Lmt.39 3 Upper Lmt 9.6 Slde 4 Confdence Interval for E( p ) The 95% confdence nterval estmate of the mean numer of cars sold when 3 TV ads are run s: =.39 to 9.6 cars Slde 4 4

15 Predcton Interval for p Ecel s Predcton Interval Output H PREDICTION INTERVAL Varance of nd Std. Dev. of nd.68 4 Margn of Error Lower Lmt Upper Lmt I Slde 43 Predcton Interval for p The 95% predcton nterval estmate of the numer of cars sold n one partcular week when 3 TV ads are run s: = 6.7 to 33.8 cars Slde 44 Resdual Analss If the assumptons aout the error term ε appear questonale, the hpothess tests aout the sgnfcance of the regresson relatonshp and the nterval estmaton results ma not e vald. The resduals provde the est nformaton aout ε. Resdual for Oservaton ˆ Much of the resdual analss s ased on an eamnaton of graphcal plots. Slde 45 5

16 Resdual Plot Aganst If the assumpton that the varance of ε s the same for all values of s vald, and the assumed regresson model s an adequate representaton of the relatonshp etween the varales, then The resdual plot should gve an overall mpresson of a horzontal and of ponts Slde 46 Resdual Plot Aganst ˆ Good Pattern Resdual Slde 47 Resdual Plot Aganst ˆ Nonconstant Varance Resdual Slde 48 6

17 Resdual Plot Aganst ˆ Model Form Not Adequate Resdual Slde 49 Resdual Plot Aganst Resduals Oservaton Predcted Cars Sold Resduals Slde 5 Resdual Plot Aganst Resduals TV Ads Resdual Plot 3 4 TV Ads Slde 5 7

18 End of Chapter 4 Slde 5 8