Simple Linear Regression Inference

Transcription

1 Simple Linear Regression Inference 1

2 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation The interpretation of the Confidence Interval (CI) is fixed to a level α. If we sample more than a set of observations, each set would probably give a di_erent OLS estimate of β and therefore different CI. (1- α)% of these intervals would include β, and only (α)% of the sets would deviate from β by more than a specified Δ. 2

3 Standardize a Gaussian variable As we have previously seen: We can standardize such statement such that: 3

4 The t-distribution However σ 2 in unknown, and it must be estimated by s 2. Where t ν is a t distribution with ν= n 2 degrees of freedom. 4

5 CI limits Hence the CI for β at a (1 α)% can be calculated as follows: The CI specifies a range of values within which the true parameter is credible to lie. The level of likelihood is specified by the choice of α. 5

6 Hypothesis Testing Step 1. Fix a significance level α Step 2. Specify the null hypothesis Step 3. Specify the alternative hypothesis Step 4. Calculate the test statistic Step 5. Determine the acceptance/rejection region 6

7 Null and Alternative Hypothesis Given a starting conjecture (null hypothesis), considered true as working hypothesis, we may verify the Hypothesis evaluating the discrepancy between the sample observations and what one would expect from the null hypothesis. If, given a specific level of significance α, the discrepancy is significant, the null hypothesis is rejected. Otherwise it cannot be statistically rejected. It is important to notice that you may never conclusively accept the null hypothesis (Falsification Theory). 7

8 Hypothesis Testing: β1 = 0 We want to test whether there is a linear relationship between X and Y at a generic α = 0.05 level of significance. The hypothesis will then be: H 0 : β1 = 0 H A : β1 0 8

9 Hypothesis Testing: β1 = 0 The test statistic as before defined: Hence we reject the null hypothesis if: 9

10 Hypothesis Testing: β1 = 0 The Golden Rule When n is large, the t-distribution tends to a normal distribution, hence if we choose a 5% significance level as above we can use an approximate rule of thumb and reject the hypothesis whenever the test statistics is greater the 2. 10

11 Hypothesis Testing: b1 = 0 P-value and Significant levels The p-value is a probability. The rule is: we reject the null hypothesis if p-value< α. 11

12 P-value example Suppose you fix α = 0.5, therefore α/2 = 0.025, and the p-value associated to the coefficient β 1 is It means that effectively there is a 1% chance that the relationship emerged randomly and a 99% chance that the relationship is real. Graphically we have: 12

13 Hypothesis Testing: β1 = c Similarly if we are testing for a specific constant value named c: H 0 : β1 = c H A : β1 c Hence the rejection criterion is: Keeping in mind the possibility of a Normal approximation for large samples. 13

14 The meaning of β 1 The parameter β 1 expresses how much change in Y follows a unitary change in X on average. if β 1 > 0 an increase in X is proportional to the increase in Y (direct relationship) if β 1 < 0 an increase in X is inversely proportional to the increase in Y (indirect relationship) 14

15 Regression and Correlation 15

16 Linear relationship The Σx i y i function measures the intensity of the linear relationship between X and Y. When this amount is expressed as mean observation contribution it is called covariance. 16

17 Linear relationship The covariance is a measure that depends on the measurement scalar. An alternative that is a relative index is the Pearson correlation coefficient. 17

18 Relationship and Dependence One must not however mistake the Pearson coefficient with the regression coefficient β 1 of a simple linear regression, as they have different formulas and different meanings. They can be linked by the following formula: Clearly the meaning also differs. The Pearson coefficient (r) measures the linear bidirectional relationship between X and Y (i.e. Y and X). 18

19 Linear relationship The regression coefficient β 1 measures the linear dependence of Y from X. Given a linear relationship, the dependence may be from X Y or Y X. In regression analysis one must ex ante decide which dependence to investigate. Correlation analysis in not concerned with causal links. Regression analysis is based of causal links. 19

20 Linear relationship Correlation does not imply causality. Example: If there are many hospital recoveries (X) in zones with many doctors (Y), it does not imply that doctors cause recoveries, but it identifies a strong linear link. 20

21 Residuals squared 21

22 Residual properties 22

23 Sum of squares TSS = Total Sum of Squares RSS = Residual Sum of Squares ESS = Explained Sum of Squares 23

24 R 2 : coefficient of determination The coefficient of determination R 2 expresses the proportion of total deviance explained by the regression model of Y on X. As one cannot explain more then all the existing deviance we can write as follows: 24

25 R 2 properties R 2 = 0 implies that the explicative contribution of the model is null, hence the deviance is completely expressed by the random component. R 2 = 1 implies that all N observations lie on the regression line, hence the whole variability is explained by the model. 25

26 R 2 properties All intermediate cases imply that the higher/lower the value of R 2 the more/less of variability is expressed by the choice of the model. A model with an R 2 = 0.80 implies that 80% of the variability is explained by the chosen model. The R 2 is said to be a fitting index as it measures the adaptiveness of the chosen model to the specified dataset. 26

27 R 2 and r Hence the coefficient of determination is equal to the coefficient of correlation squared. Another equally simple and efficient relationship is can be obtained by: 27

28 ANalysis Of VAriance The total variability decomposition shows the error component and the model component. We also know that: 28

29 ANalysis Of VAriance As the square of a Normal distribution is a Chi-squared distribution: Omitting here the proof for: The ratio of χ 2 divided by the d.f. is known to be an F distribution, as follows: 29

30 ANalysis Of VAriance One may then verify the null hypothesis H 0 : β 1 = 0 with respect to H A : β 1 0 with the following test statistic: A strong linear relationship between X and Y will determine a high valued test statistic, which supports the model, and reject the null hypothesis. Therefore: * F F 1; n 2 H0 : 1 0 is rejected empirical value theoretical value 30

31 Anova table 31

32 Forecast Often the estimated regression model is used to predict the expected Y value (i.e ˆ Y ) which corresponds to a specific value of the X variable in this case indicated with Xκ. The standard error of such value is equal to: 32

33 Forecast The 1 α confidence intervals for such value are then specified as: We may notice that the value of the s.e. increases proportionally to the distance between Xκ and X, therefore taking distant forecasts will worsen the quality of the estimate. It can also happen that the linear relation breaks down for very high or low Xκ values, as it is limited to the observe scatter plot. In this case taking a forecast outside the range of the observed values can be misleading. 33

34 Residual analysis The residual analysis is a graphical approach used to evaluate if the assumptions of the model are valid and thus to determine whether the regression model selected is appropriate. The assumption that can be graphically evaluated are: Linearity; Independence of errors; Normality of errors; Equal variance. 34

35 Linearity To evaluate linearity, plot the residuals on the vertical axis against the corresponding Xi values on the horizontal axis. If the linear model is appropriate fro the data, there will be no apparent pattern in this plot. 35

36 Independence For data collected over time, residuals can be plotted versus the time dimension to search for a relationship between consecutive residuals. Normality It can be evaluated with a normal probability plot that compare the actual versus the theoretical values of the residuals. 36

37 Equal variance Plot residuals versus X i. 37

38 Business Exercises Managerial decisions are often based on the relationship between two or more variables. For example, after considering the relationship between advertising expenditures and sales, a marketing manager might attempt to predict sales for a given level of advertising expenditure. A public utility might use the relationship between daily temperature and the demand for electricity to predict electricity consumption on next months temperature forecasts. 38

39 Business Exercises A candy bar manufacturer is interested in trying to estimate how sales are influenced by the price of their product. To do this the company randomly chooses six small cities and offers the candy bar at different prices. Using the candy bar sales as the dependent variable, the company will conduct a simple linear regression on the data below: 39

40 Business Exercises: ˆ ˆ 0, 1 40

41 Business Exercises: TSS 41

42 Business Exercises: TSS 42

43 Business Exercises: ESS 43

44 Numerical Example 44

45 45

46 46

47 Therefore one can conclude that 95 times out of 100 the true value β 1 lies between 0.25 and

48 The linear relationship is very strong, 97%. As an alternative to the F-test one can present the t-test. Which implies that the variable of circulating vehicles is significant in the regression. 48