Regression analysis. Advanced Financial Accounting II Åbo Akademi School of Business

Transcription

1 Regression analysis Advanced Financial Accounting II Åbo Akademi School of Business

2 Regression analysis A statistical process for estimating the relationships among variables Includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables Helps one understand how the typical value of the dependent variable (or 'Criterion Variable') changes when any one of the independent variables is varied, while the other independent variables are held fixed

3 Regression models and regression function Regression models involve the following variables: The unknown parameters, b, which may represent a scalar or a vector. The independent variables, X. The dependent variable, Y. A regression model relates Y to a function of X and b Y = f(x,b)

4 Regression model and regression function... Regression analysis estimates the conditional expectation of the dependent variable given the independent variables E(Y X) = f(x,b) The estimation target is the regression function Y = f(x,b) it is also of interest to characterize the variation of the dependent variable around the regression function, which can be described by a probability distribution

5 Linear regression In linear regression, the model specification is that the dependent variable, is a linear combination of the parameters b need not be linear in the independent variables X For example, in simple linear regression for modeling n data points there is one independent variable X, and two parameters, b 0 and b 1 giving the straight line y i = b 0 + b 1 x i + e i e i is an error term and the subscript i indexes a particular observation

6 Simple linear regression Example of simple linear regression, which has one independent variable

7 Diagnostics Once a regression model has been constructed, it may be important to confirm the goodness of fit of the model and the statistical significance of the estimated parameters Commonly used checks of goodness of fit include the coefficient of determination R 2 analyses of the pattern of residuals hypothesis testing Statistical significance can be checked by F-test of the overall fit t-tests of individual parameters

8 Goodness of fit Coefficient of determination R 2 The coefficient of determination, R 2 indicates how well data points fit a line or curve Provides a measure of how well observed outcomes are replicated by the model, as the proportion of total variation of outcomes explained by the model R 2 SS SS SS 1 SS res res i i res tot y i,where f 2, the residualsumof squares 2 y y, the total sumof squares i i The better the linear regression fits the data, the closer the value of R 2 is to one

9 Goodness of fit Adjusted R 2 R 2 automatically increases when extra explanatory variables are added to the model Some of the increase may be due to spurious effects A modification of R 2 adjusts for the number of explanatory terms in a model relative to the number of data points Unlike R 2, the adjusted R 2 increases when a new explanator is included only if the new explanator improves the R 2 more than would be expected in the absence of any explanatory value being added by the new explanator

10 Simple linear regression analysis an example Research question: Does the amount of money spent on advertising in affect the yearly sales of a company? Data: File: AFAII_Regression_Excercise.xlsx Yearly sales (Sales) Amount spent on advertising (AdvTotal) for 100 companies Regression equation to estimate: Sales i = b 0 + b 1 AdvTotal i + e i

11 Simple regression analysis with SPSS Analyze Regression Linear Move Sales to Dependent Move AdvTotal to Independent(s) OK

12 Simple Linear Regression Analysis with SPSS Interpretation Model fit Adjusted R 2 = % of the variation in the yearly sales is explained by the amount spent on advertising all other factors fixed

13 Simple Linear Regression Analysis with SPSS Significance of total model The F-statistics for the total model significant at 5 % level

14 Simple Linear Regression Analysis with SPSS Interpretation Coefficients Estimated regression equation t-values for both Constant and the independent variable AdvTotal > 1.96 the parameter estimates are significant at 5 % level Sales i = , AdvTotal i + e i

15 Multiple linear regression analysis In the more general multiple regression model, there are p independent variables: y i = b 0 + b 1 x i1 + b 2 x i2 + + b p x ip + e i The predictor variables have to be linearly independent, i.e. it is not possible to express any predictor as a linear combination of the others Highly correlated predictor variables lead to multicollinearity problems where the coefficient estimates may change erratically in response to small changes in the model or the data Multicollinearity does not reduce the predictive power or reliability of the model as a whole but it may not give valid results about any individual predictor

16 Multiple linear regression analysis an example Research question: Do the amounts of money spent on advertising in TV, web, and press affect the yearly sales of a company? Data: File: AFAII_Regression_Excercise.xlsx Yearly sales (Sales) Amount spent on advertising in TV (AdvTV) Amount spent on advertising in web (AdvWeb) Amount spent on advertising in press (AdvPress) for 100 companies Regression equation to estimate: Sales i = b 0 + b 1 AdvTV i + b 2 AdvWeb i + b 3 AdvPress i + e i

17 Multiple linear regression analysis with SPSS Analyze Regression Linear Move Sales to Dependent Move AdvTV, AdvWeb, and AdvPress to Independent(s) Method: Enter OK

18 MLR with SPSS Interpretation Coefficients for all three independent variables are estimated

19 MLR with SPSS Interpretation Goodness of fit Adjusted R 2 = % of the variation in the yearly sales is explained by the amount spent on advertising in TV, web and press

20 MLR with SPSS Interpretation Significance of total model The F-statistics for the total model significant at 5 % level

21 MLR with SPSS Interpretation Coefficients Coefficients for AdvTV and AdvWeb significant at 5 % level (t-value > 1.96, significance > 0.05) Constant and coefficient for AdvPress insignificant

22 Stepwise regression models The method Enter estimates a model simultaneously including all the suggested variables that pass some predefined criteria The insignificance of one of the suggested predictor variables, AdvPress, suggests that a more suitable model could be found by eliminating this variable In order to find a suitable variable combination, a stepwise estimation process may be selected In SPSS: Method: Stepwise

23 Stepwise MLR with SPSS The variables AdvTV and AdvWeb were entered in the regression model in the order they improve the total model significance (Fstatistics). AdvPress was left outside the model.

24 Stepwise MLR with SPSS Development of Goodness of fit Entering the second independent variable AdvWeb increases the explanation power of the model from 34.9 % to 39.4 %

25 Stepwise MLR with SPSS Coefficients Estimated regression equation t-values for both Constant and the independent variables AdvTV and AdvWeb > 1.96 the parameter estimates are significant at 5 % level Sales i = AdvTV i AdvWeb i + e i