Section 1: Simple Linear Regression

Size: px
Start display at page:

Download "Section 1: Simple Linear Regression"

Transcription

1 Section 1: Simple Linear Regression Carlos M. Carvalho The University of Texas McCombs School of Business 1

2 Regression: General Introduction Regression analysis is the most widely used statistical tool for understanding relationships among variables It provides a conceptually simple method for investigating functional relationships between one or more factors and an outcome of interest The relationship is expressed in the form of an equation or a model connecting the response or dependent variable and one or more explanatory or predictor variable 2

3 Why? Straight prediction questions: For who much will my house sell? How many runs per game will the Red Sox score in 2011? Will this person like that movie? (Netflix prize) Explanation and understanding: What is the impact of MBA on income? How does the returns of a mutual fund relates to the market? Does Wallmart discriminates against women regarding salaries? 3

4 1st Example: Predicting House Prices Problem: Predict market price based on observed characteristics Solution: Look at property sales data where we know the price and some observed characteristics. Build a decision rule that predicts price as a function of the observed characteristics. 4

5 Predicting House Prices What characteristics do we use? We have to define the variables of interest and develop a specific quantitative measure of these variables Many factors or variables affect the price of a house size number of baths garage, air conditioning, etc neighborhood 5

6 Predicting House Prices To keep things super simple, let s focus only on size. The value that we seek to predict is called the dependent (or output) variable, and we denote this: Y = price of house (e.g. thousands of dollars) The variable that we use to guide prediction is the explanatory (or input) variable, and this is labelled X = size of house (e.g. thousands of square feet) 6

7 Predicting House Prices What does this data look like? 7

8 Predicting House Prices It is much more useful to look at a scatterplot price In other words, view the data as points in the X Y plane. size 8

9 Regression Model Y = response or outcome variable X 1, X 2, X 3,..., Xp = explanatory or input variables The general relationship approximated by: Y = f (X 1, X 2,..., X p ) + e And a linear relationship is written Y = b 0 + b 1 X 1 + b 2 X b p X p + e 9

10 Linear Prediction Appears to be a linear relationship between price and size: As size goes up, price goes up. The line shown was fit by the eyeball method. 10

11 Linear Prediction Recall that the equation of a line is: Y = b 0 + b 1 X Where b 0 is the intercept and b 1 is the slope. The intercept value is in units of Y ($1,000). The slope is in units of Y per units of X ($1,000/1,000 sq ft). 11

12 Linear Prediction Y b 1 Y = b 0 + b 1 X b X Our eyeball line has b 0 = 35, b 1 =

13 Linear Prediction We can now predict the price of a house when we know only the size; just read the value off the line that we ve drawn. For example, given a house with of size X = 2.2. Predicted price Ŷ = (2.2) = 123. Note: Conversion from 1,000 sq ft to $1,000 is done for us by the slope coefficient (b 1 ) 13

14 Linear Prediction Can we do better than the eyeball method? We desire a strategy for estimating the slope and intercept parameters in the model Ŷ = b 0 + b 1 X A reasonable way to fit a line is to minimize the amount by which the fitted value differs from the actual value. This amount is called the residual. 14

15 Linear Prediction What is the fitted value? Y i Ŷ i X i The dots are the observed values and the line represents our fitted values given by Ŷ i = b 0 + b 1 X 1. 15

16 Linear Prediction What is the residual for the ith observation? Y i Ŷ i e i = Y i Ŷ i = Residual i X i We can write Y i = Ŷi + (Y i Ŷi) = Ŷ i + e i. 16

17 Least Squares Ideally we want to minimize the size of all residuals: If they were all zero we would have a perfect line. Trade-off between moving closer to some points and at the same time moving away from other points. The line fitting process: Give weights to all of the residuals. Minimize the total of residuals to get best fit. Least Squares chooses b 0 and b 1 to minimize N i=1 e2 i N ei 2 = e1+e en 2 = (Y 1 Ŷ1) 2 +(Y 2 Ŷ2) 2 + +(Y N ŶN) 2 i=1 17

18 Least Squares LS chooses a different line from ours: b 0 = and b 1 = What do b 0 and b 1 mean again? Our line LS line 18

19 Eyeball vs. LS Residuals eyeball: b 0 = 35, b 1 = 40 LS: b 0 = 38.88, b 1 = Size Price yhat-eyeball yhat-ls e-eyeball e-ls e2-eyeball e2-ls sum b0-eyeball b1-eyeball b0-ls b1-ls

20 Least Squares Excel Output SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 15 ANOVA df SS MS F Significance F Regression E-06 Residual Total Coefficients Standard Error t Stat P-value Lower 95% Intercept Size E Upper 95%

21 2nd Example: Offensive Performance in Baseball 1. Problems: 2. Solutions: Evaluate/compare traditional measures of offensive performance Help evaluate the worth of a player Compare prediction rules that forecast runs as a function of either AVG (batting average), SLG (slugging percentage) or OBP (on base percentage) 21

22 2nd Example: Offensive Performance in Baseball 22

23 Baseball Data Using AVG Each observation corresponds to a team in MLB. Each quantity is the average over a season. Y = runs per game; X = AVG (average) LS fit: Runs/Game = AVG 23

24 Baseball Data Using SLG Y = runs per game X = SLG (slugging percentage) LS fit: Runs/Game = SLG 24

25 Baseball Data Using OBP Y = runs per game X = OBP (on base percentage) LS fit: Runs/Game = OBP 25

26 Baseball Data What is the best prediction rule? Let s compare the predictive ability of each model using the average squared error 1 N N ei 2 = i=1 N i=1 ( Runs i Runs i ) 2 N 26

27 Place your Money on OBP Average Squared Error AVG SLG OBP

28 Linear Prediction Ŷ i = b 0 + b 1 X i b 0 is the intercept and b 1 is the slope We find b 0 and b 1 using Least Squares 28

29 The Least Squares Criterion The formulas for b 0 and b 1 that minimize the least squares criterion are: b 1 = r xy s y s x b 0 = Ȳ b 1 X where, X and Ȳ are the sample mean of X and Y corr(x, y) = r xy is the sample correlation s x and s y are the sample standard deviation of X and Y 29

30 Sample Mean and Sample Variance Sample Mean: measure of centrality Ȳ = 1 n n i=1 Y i Sample Variance: measure of spread s 2 y = 1 n 1 Sample Standard Deviation: s y = n ( Yi Ȳ ) 2 i=1 s 2 y 30

31 Example s 2 y = 1 n 1 n ( Yi Ȳ ) 2 i=1 X (X i X) X sample Y (Y i Ȳ ) Ȳ sample s x = 9.7 s y =

32 Covariance Measure the direction and strength of the linear relationship between Y and X n i=1 Cov(Y, X ) = (Y i Ȳ )(X i X ) n 1 (Yi Ȳ )(Xi X) < 0 (Yi Ȳ )(Xi X) > 0 Y Ȳ (Yi Ȳ )(Xi X) > X X (Yi Ȳ )(Xi X) < 0 s y = 15.98, s x = 9.7 Cov(X, Y ) = How do we interpret that? 32

33 Correlation Correlation is the standardized covariance: corr(x, Y ) = cov(x, Y ) sx 2 sy 2 = cov(x, Y ) s x s y The correlation is scale invariant and the units of measurement don t matter: It is always true that 1 corr(x, Y ) 1. This gives the direction (- or +) and strength (0 1) of the linear relationship between X and Y. 33

34 Correlation corr(y, X ) = cov(x, Y ) s 2 x s 2 y = cov(x, Y ) s x s y = = X Y (Yi Ȳ )(Xi X) > 0 (Yi Ȳ )(Xi X) < 0 (Yi Ȳ )(Xi X) > 0 (Yi Ȳ )(Xi X) < 0 X Ȳ 34

35 Correlation corr = corr = corr = corr =

36 Correlation Only measures linear relationships: corr(x, Y ) = 0 does not mean the variables are not related corr = 0.01 corr = Also be careful with influential observations. Excel Break: correl, stdev,... 36

37 Back to Least Squares 1. Intercept: b 0 = Ȳ b 1 X Ȳ = b 0 + b 1 X The point ( X, Ȳ ) is on the regression line Least squares finds the point of means and rotate the line through that point until getting the right slope 2. Slope: b 1 = corr(x, Y ) s Y s X = n i=1 (X i X )(Y i Ȳ ) n i=1 (X i X ) 2 = Cov(X, Y ) var(x ) So, the right slope is the correlation coefficient times a scaling factor that ensures the proper units for b 1 37

38 More on Least Squares From now on, terms fitted values (Ŷ i ) and residuals (e i ) refer to those obtained from the least squares line. The fitted values and residuals have some special properties. Lets look at the housing data analysis to figure out what these properties are... 38

39 The Fitted Values and X Fitted Values corr(y.hat, x) = X 39

40 The Residuals and X Residuals corr(e, x) = 0 mean(e) = X 40

41 Why? What is the intuition for the relationship between Ŷ and e and X? Lets consider some crazy alternative line: Y Crazy line: X LS line: X X 41

42 Fitted Values and Residuals This is a bad fit We are underestimating the value of small houses and overestimating the value of big houses. Crazy Residuals corr(e, x) = -0.7 mean(e) = Clearly, we have left some predictive ability on the table X 42

43 Summary: LS is the best we can do As long as the correlation between e and X is non-zero, we could always adjust our prediction rule to do better. We need to exploit all of the predictive power in the X values and put this into Ŷ, leaving no Xness in the residuals. In Summary: Y = Ŷ + e where: Ŷ is made from X ; corr(x, Ŷ ) = 1. e is unrelated to X ; corr(x, e) = 0. 43

44 Decomposing the Variance How well does the least squares line explain variation in Y? Remember that Y = Ŷ + e Since Ŷ and e are uncorrelated, i.e. corr(ŷ, e) = 0, var(y ) = var(ŷ + e) = var(ŷ ) + var(e) n i=1 (Y i Ȳ ) 2 n i=1 = (Ŷ i Ŷ ) 2 n i=1 + (e i ē) 2 n 1 n 1 n 1 Given that ē = 0, and Ŷ = Ȳ (why?) we get to: n (Y i Ȳ ) 2 = i=1 n (Ŷ i Ȳ ) 2 + n i=1 i=1 e 2 i 44

45 Decomposing the Variance SSR: Variation in Y explained by the regression line. SSE: Variation in Y that is left unexplained. SSR = SST perfect fit. Be careful of similar acronyms; e.g. SSR for residual SS. 45

46 A Goodness of Fit Measure: R 2 The coefficient of determination, denoted by R 2, measures goodness of fit: R 2 = SSR SST = 1 SSE SST 0 < R 2 < 1. The closer R 2 is to 1, the better the fit. 46

47 Back to the House Data Back to the House Data ''- ''/ ''. Applied Regression Analysis Carlos M. Carvalho R 2 = SSR SST = 0.82 = ""#$%%&$'()*"$+, 47

48 Back to Baseball Three very similar, related ways to look at a simple linear regression... with only one X variable, life is easy R 2 corr SSE OBP SLG AVG

49 Prediction and the Modeling Goal A prediction rule is any function where you input X and it outputs Ŷ as a predicted response at X. The least squares line is a prediction rule: Ŷ = f (X ) = b 0 + b 1 X 49

50 Prediction and the Modeling Goal Ŷ is not going to be a perfect prediction. We need to devise a notion of forecast accuracy. 50

51 Prediction and the Modeling Goal There are two things that we want to know: What value of Y can we expect for a given X? How sure are we about this forecast? Or how different could Y be from what we expect? Our goal is to measure the accuracy of our forecasts or how much uncertainty there is in the forecast. One method is to specify a range of Y values that are likely, given an X value. Prediction Interval: probable range for Y-values given X 51

52 Prediction and the Modeling Goal Key Insight: To construct a prediction interval, we will have to assess the likely range of error values corresponding to a Y value that has not yet been observed We will build a probability model (e.g., normal distribution). Then we can say something like with 95% probability the error will be no less than -$28,000 or larger than $28,000. We must also acknowledge that the fitted line may be fooled by particular realizations of the residuals. 52

53 Prediction and the Modeling Goal Suppose you only had the purple points in the graph. The dashed line fits the purple points. The solid line fits all the points. Which line is better? Why? RPG AVG In summary, we need to work with the notion of a true line and a probability distribution that describes deviation around the line. 53

54 The Simple Linear Regression Model The power of statistical inference comes from the ability to make precise statements about the accuracy of the forecasts. In order to do this we must invest in a probability model. Simple Linear Regression Model: Y = β 0 + β 1 X + ε ε N(0, σ 2 ) β 0 + β 1 X represents the true line ; The part of Y that depends on X. The error term ε is independent idosyncratic noise ; The part of Y not associated with X. 54

55 The Simple Linear Regression Model Y = β 0 + β 1 X + ε y The conditional distribution for Y given X is Normal: x Y X = x N(β 0 + β 1 x, σ 2 ). 55

56 The Simple Linear Regression Model Example You are told (without looking at the data) that β 0 = 40; β 1 = 45; σ = 10 and you are asked to predict price of a 1500 square foot house. What do you know about Y from the model? Y = (1.5) + ε = ε Thus our prediction for price is Y X = 1.5 N(107.5, 10 2 ) and a 95% Prediction Interval for Y is 87.5 < Y <

57 Summary of Simple Linear Regression Assume that all observations are drawn from our regression model and that errors on those observations are independent. The model is Y i = β 0 + β 1 X i + ε i where ε is independent and identically distributed N(0, σ 2 ). independence means that knowing ε i doesn t affect your views about ε j identically distributed means that we are using the same normal for every ε i 57

58 Summary of Simple Linear Regression The model is ε i N(0, σ 2 ). Y i = β 0 + β 1 X i + ε i The SLR has 3 basic parameters: β 0, β 1 (linear pattern) σ (variation around the line). Assumptions: independence means that knowing ε i doesn t affect your views about ε j identically distributed means that we are using the same normal for every ε i 58

59 Estimation for the SLR Model SLR assumes every observation in the dataset was generated by the model: Y i = β 0 + β 1 X i + ε i This is a model for the conditional distribution of Y given X. We use Least Squares to estimate β 0 and β 1 : ˆβ 1 = b 1 = r xy s y s x ˆβ 0 = b 0 = Ȳ b 1 X 59

60 Estimation of Error Variance We estimate σ 2 with: s 2 = 1 n 2 n i=1 e 2 i = SSE n 2 (2 is the number of regression coefficients; i.e. 2 for β 0 and β 1 ). We have n 2 degrees of freedom because 2 have been used up in the estimation of b 0 and b 1. We usually use s = SSE/(n 2), in the same units as Y. It s also called the regression standard error. 60

61 Estimation of Error Variance Estimation of 2,"-"$).$s )/$0,"$123"($ Where is s in the Excel output?. Remember that whenever you see standard error read it as ).$0,"$.0?/*?-*$*"<)?0)4/ estimated standard deviation: σ is the standard deviation. Applied Regression Analysis Carlos M. Carvalho ""#$%%%&$'()*"$+ 61

62 One Picture Summary of SLR The plot below has the house data, the fitted regression line (b 0 + b 1 X ) and ±2 s... From this picture, what can you tell me about β 0, β 1 and σ 2? How about b 0, b 1 and s 2? price size 62

63 Sampling Distribution of Least Squares Estimates How much do our estimates depend on the particular random sample that we happen to observe? Imagine: Randomly draw different samples of the same size. For each sample, compute the estimates b 0, b 1, and s. If the estimates don t vary much from sample to sample, then it doesn t matter which sample you happen to observe. If the estimates do vary a lot, then it matters which sample you happen to observe. 63

64 The Importance of Understanding Variation When estimating a quantity, it is vital to develop a notion of the precision of the estimation; for example: estimate the slope of the regression line estimate the value of a house given its size estimate the expected return on a portfolio estimate the value of a brand name estimate the damages from patent infringement Why is this important? We are making decisions based on estimates, and these may be very sensitive to the accuracy of the estimates 64

65 Sampling Distribution of Least Squares Estimates 65

66 Sampling Distribution of Least Squares Estimates 66

67 Sampling Distribution of b 1 The sampling distribution of b 1 describes how estimator b 1 = ˆβ 1 varies over different samples with the X values fixed. It turns out that b 1 is normally distributed (approximately): b 1 N(β 1, sb 2 1 ). b 1 is unbiased: E[b 1 ] = β 1. s b1 is the standard error of b 1. In general, the standard error is the standard deviation of an estimate. It determines how close b 1 is to β 1. This is a number directly available from the regression output. 67

68 Sampling Distribution of b 1 Can we intuit what should be in the formula for s b1? How should s figure in the formula? What about n? Anything else? s 2 b 1 = s 2 (Xi X ) 2 = s 2 (n 1)s 2 x Three Factors: sample size (n), error variance (s 2 ), and X -spread (s x ). 68

69 Sampling Distribution of b 0 The intercept is also normal and unbiased: b 0 N(β 0, s 2 b 0 ). s 2 b 0 = var(b 0 ) = s 2 ( 1 n + X 2 (n 1)s 2 x ) What is the intuition here? 69

70 Example: Runs per Game and AVG blue line: all points red line: only purple points Which slope is closer to the true one? How much closer? RPG AVG 70

71 Example: Runs per Game and AVG Regression with all points SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 30 ANOVA df SS MS F Significance F Regression E- 07 Residual Total Coefficients Standard Error t Stat P- value Lower 95% Upper 95% Intercept AVG E s b1 =

72 Example: Runs per Game and AVG Regression with subsample SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 5 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P- value Lower 95% Upper 95% Intercept AVG s b1 =

73 Example: Runs per Game and AVG b 1 N(β 1, sb 2 1 ) Suppose β 1 = 35 blue line: N(35, ); red line: N(35, ) (b 1 β 1 ) ±2 s b b1 73

74 Confidence Intervals Since b 1 N(β 1, s 2 b 1 ), Thus: 68% Confidence Interval: b 1 ± 1 s b1 95% Confidence Interval: b 1 ± 2 s b1 99% Confidence Interval: b 1 ± 3 s b1 Same thing for b 0 95% Confidence Interval: b 0 ± 2 s b0 The confidence interval provides you with a set of plausible values for the parameters 74

75 Example: Runs per Game and AVG Regression with all points SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 30 ANOVA df SS MS F Significance F Regression E- 07 Residual Total Coefficients Standard Error t Stat P- value Lower 95% Upper 95% Intercept AVG E [b 1 2 s b1 ; b s b1 ] [23.77; 43.36] 75

76 Testing Suppose we want to assess whether or not β 1 equals a proposed value β1 0. This is called hypothesis testing. Formally we test the null hypothesis: H 0 : β 1 = β1 0 vs. the alternative H 1 : β 1 β1 0 76

77 Testing That are 2 ways we can think about testing: 1. Building a test statistic... the t-stat, t = b 1 β 0 1 s b1 This quantity measures how many standard deviations the estimate (b 1 ) from the proposed value (β 0 1 ). If the absolute value of t is greater than 2, we need to worry (why?)... we reject the hypothesis. 77

78 Testing 2. Looking at the confidence interval. If the proposed value is outside the confidence interval you reject the hypothesis. Notice that this is equivalent to the t-stat. An absolute value for t greater than 2 implies that the proposed value is outside the confidence interval... therefore reject. This is my preferred approach for the testing problem. You can t go wrong by using the confidence interval 78

79 Example: Mutual Funds Another Example of Conditional Distributions Let s investigate the performance of the Windsor Fund, an aggressive large cap fund by Vanguard... -"./0$(11#$2.$2$032.."45426$17$.8"$*2.29$ Applied Regression Analysis ""#$%%&$'()*"$+, The Carlos plot M. Carvalho shows monthly returns for Windsor vs. the S&P500 79

80 Example: Mutual Funds Consider a CAPM regression for the Windsor mutual fund. r w = β 0 + β 1 r sp500 + ɛ Let s first test β 1 = 0 H 0 : β 1 = 0. Is the Windsor fund related to the market? H 1 : β

81 Hypothesis Testing Windsor Fund Example Example: -"./(($01"$)2*345$5"65"33)42$72$8$,9:;< Mutual Funds Applied Regression Analysis Carlos M. Carvalho b s b b s b t = reject β 1 = 0 ""#$%%%&$'()*"$+, the 95% confidence interval is [0.87; 0.99]... again, reject 81

82 Example: Mutual Funds Now let s test β 1 = 1. What does that mean? H 0 : β 1 = 1 Windsor is as risky as the market. H 1 : β 1 1 and Windsor softens or exaggerates market moves. We are asking whether or not Windsor moves in a different way than the market (e.g., is it more conservative?). 82

83 -"./(($01"$)2*345$5"65"33)42$72$8$,9:;< Example: Mutual Funds Applied Regression Analysis Carlos M. Carvalho b s b b s b t = b 1 1 s b1 = = reject. ""#$%%%&$'()*"$+, the 95% confidence interval is [0.87; 0.99]... again, reject, but... 83

84 Testing Why I like Conf. Int. Suppose in testing H 0 : β 1 = 1 you got a t-stat of 6 and the confidence interval was [ , ] Do you reject H 0 : β 1 = 1? Could you justify that to you boss? Probably not (why?) 84

85 Testing Why I like Conf. Int. Now, suppose in testing H 0 : β 1 = 1 you got a t-stat of and the confidence interval was [ 100, 100] Do you accept H 0 : β 1 = 1? Could you justify that to you boss? Probably not (why?) The Confidence Interval is your best friend when it comes to testing 85

86 P-values The p-value provides a measure of how weird your estimate is if the null hypothesis is true Small p-values are evidence against the null hypothesis In the AVG vs. R/G example... H 0 : β 1 = 0. How weird is our estimate of b 1 = 33.57? Remember: b 1 N(β 1, sb 2 1 )... If the null was true (β 1 = 0), b 1 N(0, sb 2 1 ) 86

87 P-values Where is in the picture below? b 1 (if β 1=0) The p-value is the probability of seeing b 1 equal or greater than in absolute terms. Here, p-value= Small p-value = bad null 87

88 P-values H 0 : β 1 = 0... p-value = 1.24E reject SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 30 ANOVA df SS MS F Significance F Regression E- 07 Residual Total Coefficients Standard Error t Stat P- value Lower 95% Upper 95% Intercept AVG E

89 P-values How about H 0 : β 0 = 0? How weird is b 0 = 3.936? b 0 (if β 0=0) The p-value (the probability of seeing b 1 equal or greater than in absolute terms) is Small p-value = bad null 89

90 P-values H 0 : β 0 = 0... p-value = we still reject, but not with the same strength. SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 30 ANOVA df SS MS F Significance F Regression E- 07 Residual Total Coefficients Standard Error t Stat P- value Lower 95% Upper 95% Intercept AVG E

91 Testing Summary Large t or small p-value mean the same thing... p-value < 0.05 is equivalent to a t-stat > 2 in absolute value Small p-value means something weird happen if the null hypothesis was true... Bottom line, small p-value REJECT Large t REJECT But remember, always look at the confidence interveal 91

92 Forecasting The conditional forecasting problem: Given covariate X f and sample data {X i, Y i } n i=1, predict the future observation y f. The solution is to use our LS fitted value: Ŷ f = b 0 + b 1 X f. This is the easy bit. The hard (and very important) part of forecasting is assessing uncertainty about our predictions. 92

93 Forecasting The most commonly used approach is to assume that β 0 b 0, β 1 b 1 and σ s... in this case, the error is just ɛ hence the 95% plug-in prediction interval is: b 0 + b 1 X f ± 2 s It s called plug-in because we just plug-in the estimates (b 0, b 1 and s) for the unknown parameters (β 0, β 1 and σ). 93

94 Forecasting Just remember that you are uncertain about b 0 and b 1 As a practical matter if the confidence intervals are big you should be careful Some statistical software will give you a larger (and correct) predictive interval. A large predictive error variance (high uncertainty) comes from Large s (i.e., large ε s). Small n (not enough data). Small s x (not enough observed spread in covariates). Large difference between X f and X. 94

95 Forecasting Price Red lines: prediction intervals Size Green lines: plug-in prediction intervals 95

96 The Importance of Considering and Reporting Uncertainty In 1997 the Red River flooded Grand Forks, ND overtopping its levees with a 54-feet crest. 75% of the homes in the city were damaged or destroyed It was predicted that the rain and the spring melt would lead to a 49-feet crest of the river. The levees were 51-feet high. The Water Services of North Dakota had explicitly avoided communicating the uncertainty in their forecasts as they were afraid the public would loose confidence in their abilities to predict such events. 96

97 The Importance of Considering and Reporting Uncertainty It turns out the prediction interval for the flood was 49ft ± 9ft leading to a 35% probability of the levees being overtopped Should we take the point prediction (49ft) or the interval as an input for a decision problem? In general, the distribution of potential outcomes are very relevant to help us make a decision 97

98 The Importance of Considering and Reporting Uncertainty The answer seems obvious in this example (and it is)... however, you see these things happening all the time as people tend to underplay uncertainty in many situations Why do people not give intervals? Because they are embarrassed Jan Hatzius, Goldman Sachs economists talking about economic forecasts... Don t make this mistake Intervals are your friend and will lead to better decisions 98

99 House Data one more time R 2 = 82% Great R 2, we are happy using this model to predict house prices, right? price size 99

100 House Data one more time But, s = 14 leading to a predictive interval width of about US$60,000 How do you feel about the model now? As a practical matter, s is a much more relevant quantity than R 2. Once again, intervals are your friend price size 100

Week 5: Multiple Linear Regression

Week 5: Multiple Linear Regression BUS41100 Applied Regression Analysis Week 5: Multiple Linear Regression Parameter estimation and inference, forecasting, diagnostics, dummy variables Robert B. Gramacy The University of Chicago Booth School

More information

Outline. Topic 4 - Analysis of Variance Approach to Regression. Partitioning Sums of Squares. Total Sum of Squares. Partitioning sums of squares

Outline. Topic 4 - Analysis of Variance Approach to Regression. Partitioning Sums of Squares. Total Sum of Squares. Partitioning sums of squares Topic 4 - Analysis of Variance Approach to Regression Outline Partitioning sums of squares Degrees of freedom Expected mean squares General linear test - Fall 2013 R 2 and the coefficient of correlation

More information

" Y. Notation and Equations for Regression Lecture 11/4. Notation:

 Y. Notation and Equations for Regression Lecture 11/4. Notation: Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

More information

5. Linear Regression

5. Linear Regression 5. Linear Regression Outline.................................................................... 2 Simple linear regression 3 Linear model............................................................. 4

More information

Chapter 13 Introduction to Linear Regression and Correlation Analysis

Chapter 13 Introduction to Linear Regression and Correlation Analysis Chapter 3 Student Lecture Notes 3- Chapter 3 Introduction to Linear Regression and Correlation Analsis Fall 2006 Fundamentals of Business Statistics Chapter Goals To understand the methods for displaing

More information

1 Simple Linear Regression I Least Squares Estimation

1 Simple Linear Regression I Least Squares Estimation Simple Linear Regression I Least Squares Estimation Textbook Sections: 8. 8.3 Previously, we have worked with a random variable x that comes from a population that is normally distributed with mean µ and

More information

Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011

Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011 Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011 Name: Section: I pledge my honor that I have not violated the Honor Code Signature: This exam has 34 pages. You have 3 hours to complete this

More information

1. The parameters to be estimated in the simple linear regression model Y=α+βx+ε ε~n(0,σ) are: a) α, β, σ b) α, β, ε c) a, b, s d) ε, 0, σ

1. The parameters to be estimated in the simple linear regression model Y=α+βx+ε ε~n(0,σ) are: a) α, β, σ b) α, β, ε c) a, b, s d) ε, 0, σ STA 3024 Practice Problems Exam 2 NOTE: These are just Practice Problems. This is NOT meant to look just like the test, and it is NOT the only thing that you should study. Make sure you know all the material

More information

Chapter 7: Simple linear regression Learning Objectives

Chapter 7: Simple linear regression Learning Objectives Chapter 7: Simple linear regression Learning Objectives Reading: Section 7.1 of OpenIntro Statistics Video: Correlation vs. causation, YouTube (2:19) Video: Intro to Linear Regression, YouTube (5:18) -

More information

1. What is the critical value for this 95% confidence interval? CV = z.025 = invnorm(0.025) = 1.96

1. What is the critical value for this 95% confidence interval? CV = z.025 = invnorm(0.025) = 1.96 1 Final Review 2 Review 2.1 CI 1-propZint Scenario 1 A TV manufacturer claims in its warranty brochure that in the past not more than 10 percent of its TV sets needed any repair during the first two years

More information

Univariate Regression

Univariate Regression Univariate Regression Correlation and Regression The regression line summarizes the linear relationship between 2 variables Correlation coefficient, r, measures strength of relationship: the closer r is

More information

Module 5: Multiple Regression Analysis

Module 5: Multiple Regression Analysis Using Statistical Data Using to Make Statistical Decisions: Data Multiple to Make Regression Decisions Analysis Page 1 Module 5: Multiple Regression Analysis Tom Ilvento, University of Delaware, College

More information

Regression step-by-step using Microsoft Excel

Regression step-by-step using Microsoft Excel Step 1: Regression step-by-step using Microsoft Excel Notes prepared by Pamela Peterson Drake, James Madison University Type the data into the spreadsheet The example used throughout this How to is a regression

More information

2. Simple Linear Regression

2. Simple Linear Regression Research methods - II 3 2. Simple Linear Regression Simple linear regression is a technique in parametric statistics that is commonly used for analyzing mean response of a variable Y which changes according

More information

Multiple Linear Regression

Multiple Linear Regression Multiple Linear Regression A regression with two or more explanatory variables is called a multiple regression. Rather than modeling the mean response as a straight line, as in simple regression, it is

More information

NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( )

NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( ) Chapter 340 Principal Components Regression Introduction is a technique for analyzing multiple regression data that suffer from multicollinearity. When multicollinearity occurs, least squares estimates

More information

17. SIMPLE LINEAR REGRESSION II

17. SIMPLE LINEAR REGRESSION II 17. SIMPLE LINEAR REGRESSION II The Model In linear regression analysis, we assume that the relationship between X and Y is linear. This does not mean, however, that Y can be perfectly predicted from X.

More information

Simple Regression Theory II 2010 Samuel L. Baker

Simple Regression Theory II 2010 Samuel L. Baker SIMPLE REGRESSION THEORY II 1 Simple Regression Theory II 2010 Samuel L. Baker Assessing how good the regression equation is likely to be Assignment 1A gets into drawing inferences about how close the

More information

Part 2: Analysis of Relationship Between Two Variables

Part 2: Analysis of Relationship Between Two Variables Part 2: Analysis of Relationship Between Two Variables Linear Regression Linear correlation Significance Tests Multiple regression Linear Regression Y = a X + b Dependent Variable Independent Variable

More information

Simple Linear Regression Inference

Simple Linear Regression Inference Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

More information

Estimation of σ 2, the variance of ɛ

Estimation of σ 2, the variance of ɛ Estimation of σ 2, the variance of ɛ The variance of the errors σ 2 indicates how much observations deviate from the fitted surface. If σ 2 is small, parameters β 0, β 1,..., β k will be reliably estimated

More information

A Review of Cross Sectional Regression for Financial Data You should already know this material from previous study

A Review of Cross Sectional Regression for Financial Data You should already know this material from previous study A Review of Cross Sectional Regression for Financial Data You should already know this material from previous study But I will offer a review, with a focus on issues which arise in finance 1 TYPES OF FINANCIAL

More information

Week TSX Index 1 8480 2 8470 3 8475 4 8510 5 8500 6 8480

Week TSX Index 1 8480 2 8470 3 8475 4 8510 5 8500 6 8480 1) The S & P/TSX Composite Index is based on common stock prices of a group of Canadian stocks. The weekly close level of the TSX for 6 weeks are shown: Week TSX Index 1 8480 2 8470 3 8475 4 8510 5 8500

More information

DEPARTMENT OF PSYCHOLOGY UNIVERSITY OF LANCASTER MSC IN PSYCHOLOGICAL RESEARCH METHODS ANALYSING AND INTERPRETING DATA 2 PART 1 WEEK 9

DEPARTMENT OF PSYCHOLOGY UNIVERSITY OF LANCASTER MSC IN PSYCHOLOGICAL RESEARCH METHODS ANALYSING AND INTERPRETING DATA 2 PART 1 WEEK 9 DEPARTMENT OF PSYCHOLOGY UNIVERSITY OF LANCASTER MSC IN PSYCHOLOGICAL RESEARCH METHODS ANALYSING AND INTERPRETING DATA 2 PART 1 WEEK 9 Analysis of covariance and multiple regression So far in this course,

More information

Section 14 Simple Linear Regression: Introduction to Least Squares Regression

Section 14 Simple Linear Regression: Introduction to Least Squares Regression Slide 1 Section 14 Simple Linear Regression: Introduction to Least Squares Regression There are several different measures of statistical association used for understanding the quantitative relationship

More information

CHAPTER 13 SIMPLE LINEAR REGRESSION. Opening Example. Simple Regression. Linear Regression

CHAPTER 13 SIMPLE LINEAR REGRESSION. Opening Example. Simple Regression. Linear Regression Opening Example CHAPTER 13 SIMPLE LINEAR REGREION SIMPLE LINEAR REGREION! Simple Regression! Linear Regression Simple Regression Definition A regression model is a mathematical equation that descries the

More information

2. What is the general linear model to be used to model linear trend? (Write out the model) = + + + or

2. What is the general linear model to be used to model linear trend? (Write out the model) = + + + or Simple and Multiple Regression Analysis Example: Explore the relationships among Month, Adv.$ and Sales $: 1. Prepare a scatter plot of these data. The scatter plots for Adv.$ versus Sales, and Month versus

More information

Testing for Lack of Fit

Testing for Lack of Fit Chapter 6 Testing for Lack of Fit How can we tell if a model fits the data? If the model is correct then ˆσ 2 should be an unbiased estimate of σ 2. If we have a model which is not complex enough to fit

More information

Example: Boats and Manatees

Example: Boats and Manatees Figure 9-6 Example: Boats and Manatees Slide 1 Given the sample data in Table 9-1, find the value of the linear correlation coefficient r, then refer to Table A-6 to determine whether there is a significant

More information

2013 MBA Jump Start Program. Statistics Module Part 3

2013 MBA Jump Start Program. Statistics Module Part 3 2013 MBA Jump Start Program Module 1: Statistics Thomas Gilbert Part 3 Statistics Module Part 3 Hypothesis Testing (Inference) Regressions 2 1 Making an Investment Decision A researcher in your firm just

More information

Simple linear regression

Simple linear regression Simple linear regression Introduction Simple linear regression is a statistical method for obtaining a formula to predict values of one variable from another where there is a causal relationship between

More information

4. Simple regression. QBUS6840 Predictive Analytics. https://www.otexts.org/fpp/4

4. Simple regression. QBUS6840 Predictive Analytics. https://www.otexts.org/fpp/4 4. Simple regression QBUS6840 Predictive Analytics https://www.otexts.org/fpp/4 Outline The simple linear model Least squares estimation Forecasting with regression Non-linear functional forms Regression

More information

Basic Statistics and Data Analysis for Health Researchers from Foreign Countries

Basic Statistics and Data Analysis for Health Researchers from Foreign Countries Basic Statistics and Data Analysis for Health Researchers from Foreign Countries Volkert Siersma siersma@sund.ku.dk The Research Unit for General Practice in Copenhagen Dias 1 Content Quantifying association

More information

Final Exam Practice Problem Answers

Final Exam Practice Problem Answers Final Exam Practice Problem Answers The following data set consists of data gathered from 77 popular breakfast cereals. The variables in the data set are as follows: Brand: The brand name of the cereal

More information

Regression Analysis: A Complete Example

Regression Analysis: A Complete Example Regression Analysis: A Complete Example This section works out an example that includes all the topics we have discussed so far in this chapter. A complete example of regression analysis. PhotoDisc, Inc./Getty

More information

Econometrics Simple Linear Regression

Econometrics Simple Linear Regression Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight

More information

Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression

Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression Objectives: To perform a hypothesis test concerning the slope of a least squares line To recognize that testing for a

More information

11. Analysis of Case-control Studies Logistic Regression

11. Analysis of Case-control Studies Logistic Regression Research methods II 113 11. Analysis of Case-control Studies Logistic Regression This chapter builds upon and further develops the concepts and strategies described in Ch.6 of Mother and Child Health:

More information

Outline: Demand Forecasting

Outline: Demand Forecasting Outline: Demand Forecasting Given the limited background from the surveys and that Chapter 7 in the book is complex, we will cover less material. The role of forecasting in the chain Characteristics of

More information

5. Multiple regression

5. Multiple regression 5. Multiple regression QBUS6840 Predictive Analytics https://www.otexts.org/fpp/5 QBUS6840 Predictive Analytics 5. Multiple regression 2/39 Outline Introduction to multiple linear regression Some useful

More information

Recall this chart that showed how most of our course would be organized:

Recall this chart that showed how most of our course would be organized: Chapter 4 One-Way ANOVA Recall this chart that showed how most of our course would be organized: Explanatory Variable(s) Response Variable Methods Categorical Categorical Contingency Tables Categorical

More information

12: Analysis of Variance. Introduction

12: Analysis of Variance. Introduction 1: Analysis of Variance Introduction EDA Hypothesis Test Introduction In Chapter 8 and again in Chapter 11 we compared means from two independent groups. In this chapter we extend the procedure to consider

More information

KSTAT MINI-MANUAL. Decision Sciences 434 Kellogg Graduate School of Management

KSTAT MINI-MANUAL. Decision Sciences 434 Kellogg Graduate School of Management KSTAT MINI-MANUAL Decision Sciences 434 Kellogg Graduate School of Management Kstat is a set of macros added to Excel and it will enable you to do the statistics required for this course very easily. To

More information

Exercise 1.12 (Pg. 22-23)

Exercise 1.12 (Pg. 22-23) Individuals: The objects that are described by a set of data. They may be people, animals, things, etc. (Also referred to as Cases or Records) Variables: The characteristics recorded about each individual.

More information

A Primer on Forecasting Business Performance

A Primer on Forecasting Business Performance A Primer on Forecasting Business Performance There are two common approaches to forecasting: qualitative and quantitative. Qualitative forecasting methods are important when historical data is not available.

More information

The importance of graphing the data: Anscombe s regression examples

The importance of graphing the data: Anscombe s regression examples The importance of graphing the data: Anscombe s regression examples Bruce Weaver Northern Health Research Conference Nipissing University, North Bay May 30-31, 2008 B. Weaver, NHRC 2008 1 The Objective

More information

Solution Let us regress percentage of games versus total payroll.

Solution Let us regress percentage of games versus total payroll. Assignment 3, MATH 2560, Due November 16th Question 1: all graphs and calculations have to be done using the computer The following table gives the 1999 payroll (rounded to the nearest million dolars)

More information

Forecasting in supply chains

Forecasting in supply chains 1 Forecasting in supply chains Role of demand forecasting Effective transportation system or supply chain design is predicated on the availability of accurate inputs to the modeling process. One of the

More information

Section A. Index. Section A. Planning, Budgeting and Forecasting Section A.2 Forecasting techniques... 1. Page 1 of 11. EduPristine CMA - Part I

Section A. Index. Section A. Planning, Budgeting and Forecasting Section A.2 Forecasting techniques... 1. Page 1 of 11. EduPristine CMA - Part I Index Section A. Planning, Budgeting and Forecasting Section A.2 Forecasting techniques... 1 EduPristine CMA - Part I Page 1 of 11 Section A. Planning, Budgeting and Forecasting Section A.2 Forecasting

More information

International Statistical Institute, 56th Session, 2007: Phil Everson

International Statistical Institute, 56th Session, 2007: Phil Everson Teaching Regression using American Football Scores Everson, Phil Swarthmore College Department of Mathematics and Statistics 5 College Avenue Swarthmore, PA198, USA E-mail: peverso1@swarthmore.edu 1. Introduction

More information

Causal Forecasting Models

Causal Forecasting Models CTL.SC1x -Supply Chain & Logistics Fundamentals Causal Forecasting Models MIT Center for Transportation & Logistics Causal Models Used when demand is correlated with some known and measurable environmental

More information

Introduction to Regression and Data Analysis

Introduction to Regression and Data Analysis Statlab Workshop Introduction to Regression and Data Analysis with Dan Campbell and Sherlock Campbell October 28, 2008 I. The basics A. Types of variables Your variables may take several forms, and it

More information

Premaster Statistics Tutorial 4 Full solutions

Premaster Statistics Tutorial 4 Full solutions Premaster Statistics Tutorial 4 Full solutions Regression analysis Q1 (based on Doane & Seward, 4/E, 12.7) a. Interpret the slope of the fitted regression = 125,000 + 150. b. What is the prediction for

More information

One-Way Analysis of Variance

One-Way Analysis of Variance One-Way Analysis of Variance Note: Much of the math here is tedious but straightforward. We ll skim over it in class but you should be sure to ask questions if you don t understand it. I. Overview A. We

More information

Good luck! BUSINESS STATISTICS FINAL EXAM INSTRUCTIONS. Name:

Good luck! BUSINESS STATISTICS FINAL EXAM INSTRUCTIONS. Name: Glo bal Leadership M BA BUSINESS STATISTICS FINAL EXAM Name: INSTRUCTIONS 1. Do not open this exam until instructed to do so. 2. Be sure to fill in your name before starting the exam. 3. You have two hours

More information

Coefficient of Determination

Coefficient of Determination Coefficient of Determination The coefficient of determination R 2 (or sometimes r 2 ) is another measure of how well the least squares equation ŷ = b 0 + b 1 x performs as a predictor of y. R 2 is computed

More information

Stata Walkthrough 4: Regression, Prediction, and Forecasting

Stata Walkthrough 4: Regression, Prediction, and Forecasting Stata Walkthrough 4: Regression, Prediction, and Forecasting Over drinks the other evening, my neighbor told me about his 25-year-old nephew, who is dating a 35-year-old woman. God, I can t see them getting

More information

Violent crime total. Problem Set 1

Violent crime total. Problem Set 1 Problem Set 1 Note: this problem set is primarily intended to get you used to manipulating and presenting data using a spreadsheet program. While subsequent problem sets will be useful indicators of the

More information

How To Run Statistical Tests in Excel

How To Run Statistical Tests in Excel How To Run Statistical Tests in Excel Microsoft Excel is your best tool for storing and manipulating data, calculating basic descriptive statistics such as means and standard deviations, and conducting

More information

2. Linear regression with multiple regressors

2. Linear regression with multiple regressors 2. Linear regression with multiple regressors Aim of this section: Introduction of the multiple regression model OLS estimation in multiple regression Measures-of-fit in multiple regression Assumptions

More information

Two-sample hypothesis testing, II 9.07 3/16/2004

Two-sample hypothesis testing, II 9.07 3/16/2004 Two-sample hypothesis testing, II 9.07 3/16/004 Small sample tests for the difference between two independent means For two-sample tests of the difference in mean, things get a little confusing, here,

More information

Introduction to Linear Regression

Introduction to Linear Regression 14. Regression A. Introduction to Simple Linear Regression B. Partitioning Sums of Squares C. Standard Error of the Estimate D. Inferential Statistics for b and r E. Influential Observations F. Regression

More information

Least Squares Estimation

Least Squares Estimation Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David

More information

Doing Multiple Regression with SPSS. In this case, we are interested in the Analyze options so we choose that menu. If gives us a number of choices:

Doing Multiple Regression with SPSS. In this case, we are interested in the Analyze options so we choose that menu. If gives us a number of choices: Doing Multiple Regression with SPSS Multiple Regression for Data Already in Data Editor Next we want to specify a multiple regression analysis for these data. The menu bar for SPSS offers several options:

More information

IAPRI Quantitative Analysis Capacity Building Series. Multiple regression analysis & interpreting results

IAPRI Quantitative Analysis Capacity Building Series. Multiple regression analysis & interpreting results IAPRI Quantitative Analysis Capacity Building Series Multiple regression analysis & interpreting results How important is R-squared? R-squared Published in Agricultural Economics 0.45 Best article of the

More information

Chapter 23. Inferences for Regression

Chapter 23. Inferences for Regression Chapter 23. Inferences for Regression Topics covered in this chapter: Simple Linear Regression Simple Linear Regression Example 23.1: Crying and IQ The Problem: Infants who cry easily may be more easily

More information

3. Regression & Exponential Smoothing

3. Regression & Exponential Smoothing 3. Regression & Exponential Smoothing 3.1 Forecasting a Single Time Series Two main approaches are traditionally used to model a single time series z 1, z 2,..., z n 1. Models the observation z t as a

More information

STAT 350 Practice Final Exam Solution (Spring 2015)

STAT 350 Practice Final Exam Solution (Spring 2015) PART 1: Multiple Choice Questions: 1) A study was conducted to compare five different training programs for improving endurance. Forty subjects were randomly divided into five groups of eight subjects

More information

Multiple Linear Regression in Data Mining

Multiple Linear Regression in Data Mining Multiple Linear Regression in Data Mining Contents 2.1. A Review of Multiple Linear Regression 2.2. Illustration of the Regression Process 2.3. Subset Selection in Linear Regression 1 2 Chap. 2 Multiple

More information

Testing Group Differences using T-tests, ANOVA, and Nonparametric Measures

Testing Group Differences using T-tests, ANOVA, and Nonparametric Measures Testing Group Differences using T-tests, ANOVA, and Nonparametric Measures Jamie DeCoster Department of Psychology University of Alabama 348 Gordon Palmer Hall Box 870348 Tuscaloosa, AL 35487-0348 Phone:

More information

Chapter 10. Key Ideas Correlation, Correlation Coefficient (r),

Chapter 10. Key Ideas Correlation, Correlation Coefficient (r), Chapter 0 Key Ideas Correlation, Correlation Coefficient (r), Section 0-: Overview We have already explored the basics of describing single variable data sets. However, when two quantitative variables

More information

Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)

Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4) Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

More information

Introducing the Linear Model

Introducing the Linear Model Introducing the Linear Model What is Correlational Research? Correlational designs are when many variables are measured simultaneously but unlike in an experiment none of them are manipulated. When we

More information

An analysis appropriate for a quantitative outcome and a single quantitative explanatory. 9.1 The model behind linear regression

An analysis appropriate for a quantitative outcome and a single quantitative explanatory. 9.1 The model behind linear regression Chapter 9 Simple Linear Regression An analysis appropriate for a quantitative outcome and a single quantitative explanatory variable. 9.1 The model behind linear regression When we are examining the relationship

More information

APPLICATION OF LINEAR REGRESSION MODEL FOR POISSON DISTRIBUTION IN FORECASTING

APPLICATION OF LINEAR REGRESSION MODEL FOR POISSON DISTRIBUTION IN FORECASTING APPLICATION OF LINEAR REGRESSION MODEL FOR POISSON DISTRIBUTION IN FORECASTING Sulaimon Mutiu O. Department of Statistics & Mathematics Moshood Abiola Polytechnic, Abeokuta, Ogun State, Nigeria. Abstract

More information

SPSS Guide: Regression Analysis

SPSS Guide: Regression Analysis SPSS Guide: Regression Analysis I put this together to give you a step-by-step guide for replicating what we did in the computer lab. It should help you run the tests we covered. The best way to get familiar

More information

One-Way Analysis of Variance (ANOVA) Example Problem

One-Way Analysis of Variance (ANOVA) Example Problem One-Way Analysis of Variance (ANOVA) Example Problem Introduction Analysis of Variance (ANOVA) is a hypothesis-testing technique used to test the equality of two or more population (or treatment) means

More information

The Assumption(s) of Normality

The Assumption(s) of Normality The Assumption(s) of Normality Copyright 2000, 2011, J. Toby Mordkoff This is very complicated, so I ll provide two versions. At a minimum, you should know the short one. It would be great if you knew

More information

HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION

HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION HOD 2990 10 November 2010 Lecture Background This is a lightning speed summary of introductory statistical methods for senior undergraduate

More information

GLM I An Introduction to Generalized Linear Models

GLM I An Introduction to Generalized Linear Models GLM I An Introduction to Generalized Linear Models CAS Ratemaking and Product Management Seminar March 2009 Presented by: Tanya D. Havlicek, Actuarial Assistant 0 ANTITRUST Notice The Casualty Actuarial

More information

Notes on Applied Linear Regression

Notes on Applied Linear Regression Notes on Applied Linear Regression Jamie DeCoster Department of Social Psychology Free University Amsterdam Van der Boechorststraat 1 1081 BT Amsterdam The Netherlands phone: +31 (0)20 444-8935 email:

More information

One-Way Analysis of Variance: A Guide to Testing Differences Between Multiple Groups

One-Way Analysis of Variance: A Guide to Testing Differences Between Multiple Groups One-Way Analysis of Variance: A Guide to Testing Differences Between Multiple Groups In analysis of variance, the main research question is whether the sample means are from different populations. The

More information

Normality Testing in Excel

Normality Testing in Excel Normality Testing in Excel By Mark Harmon Copyright 2011 Mark Harmon No part of this publication may be reproduced or distributed without the express permission of the author. mark@excelmasterseries.com

More information

Business Valuation Review

Business Valuation Review Business Valuation Review Regression Analysis in Valuation Engagements By: George B. Hawkins, ASA, CFA Introduction Business valuation is as much as art as it is science. Sage advice, however, quantitative

More information

Factors affecting online sales

Factors affecting online sales Factors affecting online sales Table of contents Summary... 1 Research questions... 1 The dataset... 2 Descriptive statistics: The exploratory stage... 3 Confidence intervals... 4 Hypothesis tests... 4

More information

Interaction between quantitative predictors

Interaction between quantitative predictors Interaction between quantitative predictors In a first-order model like the ones we have discussed, the association between E(y) and a predictor x j does not depend on the value of the other predictors

More information

MULTIPLE LINEAR REGRESSION ANALYSIS USING MICROSOFT EXCEL. by Michael L. Orlov Chemistry Department, Oregon State University (1996)

MULTIPLE LINEAR REGRESSION ANALYSIS USING MICROSOFT EXCEL. by Michael L. Orlov Chemistry Department, Oregon State University (1996) MULTIPLE LINEAR REGRESSION ANALYSIS USING MICROSOFT EXCEL by Michael L. Orlov Chemistry Department, Oregon State University (1996) INTRODUCTION In modern science, regression analysis is a necessary part

More information

Section 1.5 Linear Models

Section 1.5 Linear Models Section 1.5 Linear Models Some real-life problems can be modeled using linear equations. Now that we know how to find the slope of a line, the equation of a line, and the point of intersection of two lines,

More information

General Regression Formulae ) (N-2) (1 - r 2 YX

General Regression Formulae ) (N-2) (1 - r 2 YX General Regression Formulae Single Predictor Standardized Parameter Model: Z Yi = β Z Xi + ε i Single Predictor Standardized Statistical Model: Z Yi = β Z Xi Estimate of Beta (Beta-hat: β = r YX (1 Standard

More information

INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA)

INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA) INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA) As with other parametric statistics, we begin the one-way ANOVA with a test of the underlying assumptions. Our first assumption is the assumption of

More information

ANOVA. February 12, 2015

ANOVA. February 12, 2015 ANOVA February 12, 2015 1 ANOVA models Last time, we discussed the use of categorical variables in multivariate regression. Often, these are encoded as indicator columns in the design matrix. In [1]: %%R

More information

Simple Predictive Analytics Curtis Seare

Simple Predictive Analytics Curtis Seare Using Excel to Solve Business Problems: Simple Predictive Analytics Curtis Seare Copyright: Vault Analytics July 2010 Contents Section I: Background Information Why use Predictive Analytics? How to use

More information

Penalized regression: Introduction

Penalized regression: Introduction Penalized regression: Introduction Patrick Breheny August 30 Patrick Breheny BST 764: Applied Statistical Modeling 1/19 Maximum likelihood Much of 20th-century statistics dealt with maximum likelihood

More information

Analysis of Bayesian Dynamic Linear Models

Analysis of Bayesian Dynamic Linear Models Analysis of Bayesian Dynamic Linear Models Emily M. Casleton December 17, 2010 1 Introduction The main purpose of this project is to explore the Bayesian analysis of Dynamic Linear Models (DLMs). The main

More information

CHAPTER 13. Experimental Design and Analysis of Variance

CHAPTER 13. Experimental Design and Analysis of Variance CHAPTER 13 Experimental Design and Analysis of Variance CONTENTS STATISTICS IN PRACTICE: BURKE MARKETING SERVICES, INC. 13.1 AN INTRODUCTION TO EXPERIMENTAL DESIGN AND ANALYSIS OF VARIANCE Data Collection

More information

How To Check For Differences In The One Way Anova

How To Check For Differences In The One Way Anova MINITAB ASSISTANT WHITE PAPER This paper explains the research conducted by Minitab statisticians to develop the methods and data checks used in the Assistant in Minitab 17 Statistical Software. One-Way

More information

Additional sources Compilation of sources: http://lrs.ed.uiuc.edu/tseportal/datacollectionmethodologies/jin-tselink/tselink.htm

Additional sources Compilation of sources: http://lrs.ed.uiuc.edu/tseportal/datacollectionmethodologies/jin-tselink/tselink.htm Mgt 540 Research Methods Data Analysis 1 Additional sources Compilation of sources: http://lrs.ed.uiuc.edu/tseportal/datacollectionmethodologies/jin-tselink/tselink.htm http://web.utk.edu/~dap/random/order/start.htm

More information

Elementary Statistics Sample Exam #3

Elementary Statistics Sample Exam #3 Elementary Statistics Sample Exam #3 Instructions. No books or telephones. Only the supplied calculators are allowed. The exam is worth 100 points. 1. A chi square goodness of fit test is considered to

More information

Introduction to Linear Regression

Introduction to Linear Regression 14. Regression A. Introduction to Simple Linear Regression B. Partitioning Sums of Squares C. Standard Error of the Estimate D. Inferential Statistics for b and r E. Influential Observations F. Regression

More information