# e = random error, assumed to be normally distributed with mean 0 and standard deviation σ

Save this PDF as:

Size: px
Start display at page:

Download "e = random error, assumed to be normally distributed with mean 0 and standard deviation σ"

## Transcription

1 1 Linear Regression 1.1 Simple Linear Regression Model The linear regression model is applied if we want to model a numeric response variable and its dependency on at least one numeric factor variable. We will later see how we can add more factors, numerical as well as categorical to the model. Because of this flexibility we find regression to be a powerful tool. But again we have to pay close attention to the assumptions made for the analysis of the model to result in meaningful results. A first order regression model (Simple Linear Regression Model) where y = β 0 + β 1 x + e y = response variable x = independent or predictor or factor variable e = random error, assumed to be normally distributed with mean 0 and standard deviation σ β 0 = intercept β 1 = slope A consequence of this model is that for a given value of x the mean of the random variable y equals the deterministic part of the model (because µ e = 0). µ y = E(y) = β 0 + β 1 x β 0 + β 1 x describes a line with y-intercept β 0 and slope β 1 1

2 In conclusion the model implies that y in average follows a line, depending on x. Since y is assumed to also underlie a random influence, not all data is expected to fall on the line, but that the line will represent the mean of y for given values of x. We will see how to use sample data to estimate the parameters of the model, how to check the usefulness of the model, and how to use the model for prediction, estimation and interpretation Fitting the Model In a first step obtain a scattergram, to check visually if a linear relationship seems to be reasonable. Example 1 Conservation Ecology study (cp. exercise on pg. 100) Researchers developed two fragmentation indices to score forests - one index measuring anthropogenic (human causes) fragmentation, and one index measuring fragmentation for natural causes. Assume the data below shows the indices (rounded) for 5 forests. noindex x i anthindex y i The diagram shows a linear relationship between the two indices (a straight line would represent the relationship pretty well). If we would include a line representing the relationship between the indices, we could find the differences between the measured values and the line for the same x value. These differences between the data points and the values on the line are called deviations. 2

3 Example 2 A marketing manager conducted a study to deterine whether there is a linear relationship between money spent on advertising and company sales. The data are shown in the table below. Advertising expenses and company sales are both in \$1000. ad expenses x i company sales y i The graph shows a positive strong linear relationship between sales and and advertisement expenses. If a relationship between two variables would have no random component, the dots would all be precisely on a line and the deviations would all be 0. deviations = error of prediction = difference between observed and predicted values. We will choose the line that results in the least squared deviations (for the sample data) as the estimation for the line representing the deterministic portion of the relationship in the population. Continue example 1: The model for the anthropogenic index (y) in dependency on the natural cause index (x) is: y = β 0 + β 1 x + e 3

4 After we established through the scattergram that a linear model seems to be appropriate, we now want to estimate the parameters of the model (based on sample data). We will get the estimated regression line ŷ = ˆβ 0 + ˆβ 1 x The hats indicate that these are estimates for the true population parameters, based on sample data. Continue example 2: The model for the company sales (y) in dependency on the advertisement expenses (x) is: y = β 0 + β 1 x + e We now want to estimate the parameters of the model (based on sample data). We will get the estimated regression line ŷ = ˆβ 0 + ˆβ 1 x Estimating the model parameters using the least squares line Once we choose ˆβ 0 and ˆβ 1, we find that the deviation of the ith value from its predicted value is y i ŷ i = y i ˆβ 0 + ˆβ 1 x i. Then the sum of squared deviations for all n data points is SSE = [y i ˆβ 0 + ˆβ 1 x i ] 2 We will choose ˆβ 0 and ˆβ 1, so that the SSE is as small as possible, and call the result the least squares estimates of the population parameters β 0 and β 1. Solving this problem mathematically we get: With SS xy = x i y i ( x i )( y i ) n and SS xx = x 2 i ( x i ) 2 n the estimate for the slope is ˆβ 1 = SS xy SS xx and the estimate for the intercept is ˆβ 0 = ȳ ˆβ 1 x. Continue example 1: x i y i x 2 i yi 2 x i y i totals

5 From these results we get: SS xy = 4859 (131)(183) 5 SS xx = 3451 (131)2 5 ˆβ 1 = ˆβ 0 = = 64.4 = 18.8 = = So, ŷ = x describes the line that results in the least total squared deviations (SSE) possible. Here SSE = (found with minitab). The intercept tells us that the anthropogenic fragmentation score would be -53 in average if the natural cause score is zero. This is probably not possible. The slope tells us that in average the anthropogenic score increases by 3.42 point for every one point increase in the natural cause index. Continue example 2: 5

6 x i y i x 2 i yi 2 x i y i totals From these results we get: SS xy = (15.8)(1634) 8 SS xx = (15.8)2 8 ˆβ 1 = ˆβ 0 = = = = = So, ŷ = x describes the line that results in the least total squared deviations (SSE) possible. Interpretation: For every extra dollar sent on advertisement the company sales increase in average by \$ When the company does not spend any money on advertisement (x = 0), then the mean sales equal \$ Use SSE = SS yy ˆβ 1 SS xy to find the SSE for this regression line. SS yy = (1634)2 8 Then SSE = (62.65) = = Built in assumptions: When introducing the model, we stated that the error shall be normally distributed with mean zero and standard deviation σ independent from the value of x. This is illustrated by > diagram The first implication of the model is that the mean of y follows a straight line in x, not a curve We write µ y = E(y) = β 0 + β 1 x The model also implies that the deviations from the line are normally distributed, where the variation from the line is constant for x. 6

7 The shape of the distribution of the deviations is normal. When using the least squares estimators introduced above we also have to assume that the measurements are independent. From this discussion we learn that another important parameter in the model (beside β 0 and β 1 ) is the standard deviation of the error: σ Estimating σ When estimating the standard deviation of the error in an ANOVA model we based it on the SSE or MSE. We will do the same in regression. For large standard deviations we should expect the deviations, that is the difference between the estimates and the measured values in the sample, to be larger than for smaller standard deviations. Because of this it is not surprising that the SSE = (ŷ i y i ) 2 is part of the estimate. In fact the best estimate for σ 2 is ˆσ 2 = s 2 = SSE n 2 where SSE = (ŷ i y i ) 2 = SS yy ˆβ 1 SS xy and SS yy = yi 2 ( y i ) 2 n then the best estimate for σ is s = SSE s 2 = n 2 It is called the estimated standard error of the regression model. Continue example 1: Let us estimate the standard deviation σ in the model for the fragmentation indices. SS yy = 6931 (183)2 5 SSE = (64.4) = = then s = =

8 is the estimated standard error in the regression model. Continue example 2: The estimate for the standard deviation σ in the model for the company sales is s = = is the estimated standard error in the regression model. For interpreting s: One should expect about 95% of the observed y values to fall within 2s of the estimated line Checking Model Utility Making Inference about the slope As the sample mean x is an estimate for the population mean µ, the sample slope ˆβ 1 is an estimate for the population slope β 1. We can now continue on the same path as we did when using sample data for drawing conclusions about a population mean: find a confidence interval, learn how to conduct a test. In order to this we have to discuss the distribution of ˆβ 1. If the assumptions of the regression model hold then ˆβ 1 is normally distributed with mean µ ˆβ1 = β 1 and standard deviation σ σ ˆβ1 = SSxx σ ˆβ1 is estimated by s ˆβ1 = s SSxx which is the estimated standard error of the least squares slope ˆβ 1. Combining all this information we find that under the assumption of the regression model is t-distributed with df = n 2. t = ˆβ 1 β 1 s/ SS xx (1 α) 100% CI for β 1 ˆβ 1 ± t s SSxx where t = (1 α/2) percentile of the t-distribution with df = n 2. Continue example 1: A 95% CI for β 1 (df=3) ± (3.182) ±

9 A 95% CI for β 1 is [ ; ]. We conclude with 95% confidence that the anthropologic index increases in average between 2.26 to 4.59 points for a one point increase in the natural cause index. Continue example 2: A 95% CI for β 1 (df=8-2=6) ± (2.447) ± A 95% CI for β 1 is [ ; ]. We conclude that with 95% confidence the sales price increases in average between \$28.1 and \$73.3 for every extra dollar spent on advertisement. Does the CI provide sufficient evidence that β 1 0? This is an important question, because if β 1 would be 0, then the variable x would contribute no information on predicting y when using a linear model. Another way of answering this question is the model utility test about β 1 1. Hypotheses: Parameter of interest β 1. test type hypotheses upper tail H 0 : β 1 0 vs. H a : β 1 > 0 lower tail H 0 : β 1 0 vs. H a : β 1 < 0 two tail H 0 : β 1 = 0 vs. H a : β 1 0 Choose α. 2. Assumptions: Regression model is appropriate 3. Test statistic: t 0 = ˆβ 1 s/ SS xx, df = n 2 4. P-value: test type P-value upper tail P (t > t 0 ) lower tail P (t < t 0 ) two tail 2P (t > abs(t 0 )) 5. Decision: If P value < α reject H 0, otherwise do not reject H Put into context Continue example 1: Test if β 1 > 0 1. Hypotheses: Parameter of interest β 1. H 0 : β 1 0 vs. H a : β 1 > 0 Choose α = Assumptions: Regression model is appropriate 9

10 3. Test statistic: t 0 = / 18.8 = 9.358, df = 3 4. P-value: upper tail P (t > t 0 ) < (table IV). 5. Decision: P value < < 0.05 = α reject H 0 6. At significance level of 5% we conclude that in average the anthropogenic index increases as the natural cause index increases. The linear regression model helps explaining the anthropogenic index. Continue example 2: Test if β 1 > 0 1. Hypotheses: Parameter of interest β 1. H 0 : β 1 0 vs. H a : β 1 > 0 Choose α = Assumptions: Regression model is appropriate 3. Test statistic: t 0 = / = 5.477, df = 6 4. P-value: upper tail P (t > 5.477) < (table IV) 5. Decision: P value < 0.05 = α reject H 0 6. At significance level of 5% we conclude that the mean sales increase, when the amount spend on advertisement increases Measuring Model Fit Correlation Coefficient The correlation coefficient r = SS xy SSxx SS yy measures the strength of the linear relationship between two variables x and y. Properties 1. 1 r 1 2. independent from units used 3. abs(r) = 1 indicates that all values fall precisely on a line 4. r > (<)0 indicates a positive (negative) relationship. 10

11 5. r 0 indicates no linear relationship (Diagrams) Continue example 1: r = (233.2) = indicates a very strong positive linear relationship between the two indices. We can interpret r as an estimate of the population correlation coefficient ρ(greek letter rho). We could use a statistic based on r, to test if ρ = 0, this would be equivalent to testing if β 1 = 0, we will skip this test for that reason. Continue example 2: r = (3813.5) = indicates a very strong positive linear relationship between the company sales and the advertisement expenses. Coefficient of Determination The coefficient of determination measures the usefulness of the model, it is the correlation coefficient squared, r 2, and a number between 0 and 1. It s wide use can be explained through it s interpretation. An alternative formula for r 2 is r 2 = SS yy SSE SS yy This formula can help to understand the interpretation of r 2. SS yy measures the spread in the y values (in the sample), and SSE measures the spread in y we can not explain through the linear relationship with x. So that SS yy SSE is the spread in y explained through the linear relationship with x. By dividing this by SS yy, we find that r 2 gives the proportion in the spread of y, that is explained through the linear relationship with x. Continue Example 1: The coefficient of determination in our example is r 2 = = % in the variation in the anthropogenic index is explained through the natural cause index, this suggests that fragmentation of the two cause are highly related and interdependent. Continue Example 2: The coefficient of determination in the sales example is r 2 = = % in the variation in the company sales can be explained explained through the linear relationship with the advertisement expenses. A high coefficient of determination indicates that the model is suitable for estimation and prediction. A more detailed way of checking model fit is the Residual Analysis. 11

12 1.1.4 Residual Analysis: Checking the Assumptions Our conclusion we draw about the population do only hold up if the assumptions we make about the population hold up. When conducting a regression analysis we assume that the regression model is appropriate, that is: 1. the deterministic part of the model is linear in x. 2. the error is normally distributed, with mean 0 and standard deviation σ (independent from x). If the assumptions are reasonable we checked by analyzing a scattergram of the 2 variables of interest. We will now see how we can use further methods based on residuals to check the assumptions. The regression residuals are the differences for each measured value of y with the estimate based on the estimated model. Let i indicate an individual measurement, then ê i = y i ŷ i = y i ( ˆβ 0 + ˆβ 1 x i ) is the residual for the ith measurement. If the model is appropriate the residuals would be measurements coming from a normal distribution with mean 0 and standard deviation σ. To check if the assumption of normality is appropriate, a QQ-plot of the residuals could be analyzed. Continue example 1: This QQ-plot shows no indication that the assumption of normality is violated. If the model is appropriate then a scattergram of x versus the residuals would show a random scatter of the points around zero, over the interval of observations of x, not showing any pattern or outliers. (Diagrams: pattern, outliers, spread depending on x) Continue example 1: Too little data for this to carry sufficient information. 12

13 Continue example 2: For this example will also use A QQ-plot to check for normality. Plot of RESI1.jpg The dots fall far off the line, but the CI included indicates no significant violation of normality. of RESI1 vs ad.jpg If the model is appropriate then a scattergram of x versus the residuals would show a random scatter of the points around zero, over the interval of observations of x, not showing any pattern or outliers. In particular the scatter would be the same for all values of the predictor. For this data this might be violated. There is lot of scatter around ad=2, but this not enough data to be conclusive. (Diagrams: pattern, outliers, spread depending on x) 13

14 1.1.5 Applying the Model Estimation and Prediction Once we established that the model we found describes the relationship properly, we can use the model for estimation: estimate mean values of y, E(y) for given values of x. For example if the natural cause index equals 22, what value should we expect for the anthropogenic index? prediction: new individual value based on the knowledge of x, For example, we score a different forest with a natural cause index of x = 22.5, what value do we predict for y = anthropogenic index. Which application will we be able to answer with higher accuracy? The key to both applications of the model is the least squares line, ŷ = ˆβ 0 + ˆβ 1 x, we found before, which gives us point estimators for both tasks. The next step is to find confidence intervals for E(y) for a given x, and for an individual value y, for a given x. In order to this we will have to discuss the distribution of ŷ for the two tasks. Sampling distribution of ŷ = ˆβ 0 + ˆβ 1 x 1. The standard deviation of the distribution of the estimator ŷ of the mean value E(y) at a specific value of x, say x p 1 σŷ = σ n + (x p x) 2 Sxx This is the standard error of ŷ. 2. The standard deviation of the distribution of the prediction error, y ŷ, for the predictor ŷ of an individual new y at a specific value of x, say x p σ (y ŷ) = σ This is the standard error of prediction. A (1 α)100% CI for E(y) at x = x p ŷ ± t s n + (x p x) 2 SS xx 1 n + (x p x) 2 SS xx A (1 α)100% Prediction Interval for y at x = x p ŷ ± t s n + (x p x) 2 SS xx Comment: Do a confidence interval, if interested in the mean of y for a certain value of x, but if you want to get a range where a future value of y might fall for a certain value of x, then do a prediction interval. 14

15 Continue example 1: The natural cause index for a forest was found to be x p = 27.5, where should we expect the anthropogenic index to fall? Find a prediction interval. For x p = 27.5 find ŷ = (27.5) = With df = 3 t = ± 3.182(2.049) ( /5)2 +, gives ± 3.182(2.049) which is ± = [35.324, ]. For a forest with natural cause index of 27.5 we expect with 95% confidence the anthropogenic index to fall between 35.3 and Estimate the mean of the anthropogenic index for forests with natural cause index of 27.5 at confidence level of 95% ± 3.182(2.049) n + (x p x) 2 gives ± 3.182(2.049) SS xx which is ± = [ , ]. With 95% confidence the mean anthropogenic index of forests with natural cause index of 27.5 falls between 38.9 and Continue example 2: If the company decides to invest \$2000 in advertisements, what are the sales they should expect in average? Estimate with 95% confidence. For x p = 2 find ŷ = (50.73)2 = With df = 6 t = ± 2.447(10.29) 1 8 (2 15.8/8)2 +, gives ± which is [196.60, ]. If the they invest \$2000 the mean company sales fall with 95% confidence between \$ and \$ We expect 95% of sales numbers to fall within the following interval, when the company invests \$2000 in advertisement. which is [178.81, ] ± 2.447(10.29) (2 15.8/8)2 +, gives ±

### 0.1 Multiple Regression Models

0.1 Multiple Regression Models We will introduce the multiple Regression model as a mean of relating one numerical response variable y to two or more independent (or predictor variables. We will see different

### Hypothesis testing - Steps

Hypothesis testing - Steps Steps to do a two-tailed test of the hypothesis that β 1 0: 1. Set up the hypotheses: H 0 : β 1 = 0 H a : β 1 0. 2. Compute the test statistic: t = b 1 0 Std. error of b 1 =

### Regression Analysis: A Complete Example

Regression Analysis: A Complete Example This section works out an example that includes all the topics we have discussed so far in this chapter. A complete example of regression analysis. PhotoDisc, Inc./Getty

### 1. What is the critical value for this 95% confidence interval? CV = z.025 = invnorm(0.025) = 1.96

1 Final Review 2 Review 2.1 CI 1-propZint Scenario 1 A TV manufacturer claims in its warranty brochure that in the past not more than 10 percent of its TV sets needed any repair during the first two years

### Regression. Name: Class: Date: Multiple Choice Identify the choice that best completes the statement or answers the question.

Class: Date: Regression Multiple Choice Identify the choice that best completes the statement or answers the question. 1. Given the least squares regression line y8 = 5 2x: a. the relationship between

### CHAPTER 13 SIMPLE LINEAR REGRESSION. Opening Example. Simple Regression. Linear Regression

Opening Example CHAPTER 13 SIMPLE LINEAR REGREION SIMPLE LINEAR REGREION! Simple Regression! Linear Regression Simple Regression Definition A regression model is a mathematical equation that descries the

### Estimation of σ 2, the variance of ɛ

Estimation of σ 2, the variance of ɛ The variance of the errors σ 2 indicates how much observations deviate from the fitted surface. If σ 2 is small, parameters β 0, β 1,..., β k will be reliably estimated

### where b is the slope of the line and a is the intercept i.e. where the line cuts the y axis.

Least Squares Introduction We have mentioned that one should not always conclude that because two variables are correlated that one variable is causing the other to behave a certain way. However, sometimes

### 17.0 Linear Regression

17.0 Linear Regression 1 Answer Questions Lines Correlation Regression 17.1 Lines The algebraic equation for a line is Y = β 0 + β 1 X 2 The use of coordinate axes to show functional relationships was

### Outline. Topic 4 - Analysis of Variance Approach to Regression. Partitioning Sums of Squares. Total Sum of Squares. Partitioning sums of squares

Topic 4 - Analysis of Variance Approach to Regression Outline Partitioning sums of squares Degrees of freedom Expected mean squares General linear test - Fall 2013 R 2 and the coefficient of correlation

### Simple linear regression

Simple linear regression Introduction Simple linear regression is a statistical method for obtaining a formula to predict values of one variable from another where there is a causal relationship between

### Regression in SPSS. Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology

Regression in SPSS Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology John P. Bentley Department of Pharmacy Administration University of

### Coefficient of Determination

Coefficient of Determination The coefficient of determination R 2 (or sometimes r 2 ) is another measure of how well the least squares equation ŷ = b 0 + b 1 x performs as a predictor of y. R 2 is computed

### Simple Linear Regression Chapter 11

Simple Linear Regression Chapter 11 Rationale Frequently decision-making situations require modeling of relationships among business variables. For instance, the amount of sale of a product may be related

### Simple Linear Regression

Inference for Regression Simple Linear Regression IPS Chapter 10.1 2009 W.H. Freeman and Company Objectives (IPS Chapter 10.1) Simple linear regression Statistical model for linear regression Estimating

### SELF-TEST: SIMPLE REGRESSION

ECO 22000 McRAE SELF-TEST: SIMPLE REGRESSION Note: Those questions indicated with an (N) are unlikely to appear in this form on an in-class examination, but you should be able to describe the procedures

### A correlation exists between two variables when one of them is related to the other in some way.

Lecture #10 Chapter 10 Correlation and Regression The main focus of this chapter is to form inferences based on sample data that come in pairs. Given such paired sample data, we want to determine whether

### Chapter 11: Two Variable Regression Analysis

Department of Mathematics Izmir University of Economics Week 14-15 2014-2015 In this chapter, we will focus on linear models and extend our analysis to relationships between variables, the definitions

### Final Exam Practice Problem Answers

Final Exam Practice Problem Answers The following data set consists of data gathered from 77 popular breakfast cereals. The variables in the data set are as follows: Brand: The brand name of the cereal

### Chapter 13 Introduction to Linear Regression and Correlation Analysis

Chapter 3 Student Lecture Notes 3- Chapter 3 Introduction to Linear Regression and Correlation Analsis Fall 2006 Fundamentals of Business Statistics Chapter Goals To understand the methods for displaing

### 1 Simple Linear Regression I Least Squares Estimation

Simple Linear Regression I Least Squares Estimation Textbook Sections: 8. 8.3 Previously, we have worked with a random variable x that comes from a population that is normally distributed with mean µ and

### Chapter 11: Linear Regression - Inference in Regression Analysis - Part 2

Chapter 11: Linear Regression - Inference in Regression Analysis - Part 2 Note: Whether we calculate confidence intervals or perform hypothesis tests we need the distribution of the statistic we will use.

### Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression

Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression Objectives: To perform a hypothesis test concerning the slope of a least squares line To recognize that testing for a

### Econometrics Simple Linear Regression

Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight

### Lecture 5: Correlation and Linear Regression

Lecture 5: Correlation and Linear Regression 3.5. (Pearson) correlation coefficient The correlation coefficient measures the strength of the linear relationship between two variables. The correlation is

### 12/31/2016. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2

PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 Understand linear regression with a single predictor Understand how we assess the fit of a regression model Total Sum of Squares

### 5. Linear Regression

5. Linear Regression Outline.................................................................... 2 Simple linear regression 3 Linear model............................................................. 4

### Using Minitab for Regression Analysis: An extended example

Using Minitab for Regression Analysis: An extended example The following example uses data from another text on fertilizer application and crop yield, and is intended to show how Minitab can be used to

### Premaster Statistics Tutorial 4 Full solutions

Premaster Statistics Tutorial 4 Full solutions Regression analysis Q1 (based on Doane & Seward, 4/E, 12.7) a. Interpret the slope of the fitted regression = 125,000 + 150. b. What is the prediction for

### E205 Final: Version B

Name: Class: Date: E205 Final: Version B Multiple Choice Identify the choice that best completes the statement or answers the question. 1. The owner of a local nightclub has recently surveyed a random

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### Simple Regression Theory II 2010 Samuel L. Baker

SIMPLE REGRESSION THEORY II 1 Simple Regression Theory II 2010 Samuel L. Baker Assessing how good the regression equation is likely to be Assignment 1A gets into drawing inferences about how close the

### Statistics for Management II-STAT 362-Final Review

Statistics for Management II-STAT 362-Final Review Multiple Choice Identify the letter of the choice that best completes the statement or answers the question. 1. The ability of an interval estimate to

### 1. The parameters to be estimated in the simple linear regression model Y=α+βx+ε ε~n(0,σ) are: a) α, β, σ b) α, β, ε c) a, b, s d) ε, 0, σ

STA 3024 Practice Problems Exam 2 NOTE: These are just Practice Problems. This is NOT meant to look just like the test, and it is NOT the only thing that you should study. Make sure you know all the material

### Exercise 1.12 (Pg. 22-23)

Individuals: The objects that are described by a set of data. They may be people, animals, things, etc. (Also referred to as Cases or Records) Variables: The characteristics recorded about each individual.

### Chapter 9. Section Correlation

Chapter 9 Section 9.1 - Correlation Objectives: Introduce linear correlation, independent and dependent variables, and the types of correlation Find a correlation coefficient Test a population correlation

### Lecture 2: Simple Linear Regression

DMBA: Statistics Lecture 2: Simple Linear Regression Least Squares, SLR properties, Inference, and Forecasting Carlos Carvalho The University of Texas McCombs School of Business mccombs.utexas.edu/faculty/carlos.carvalho/teaching

### SIMPLE LINEAR CORRELATION. r can range from -1 to 1, and is independent of units of measurement. Correlation can be done on two dependent variables.

SIMPLE LINEAR CORRELATION Simple linear correlation is a measure of the degree to which two variables vary together, or a measure of the intensity of the association between two variables. Correlation

### , then the form of the model is given by: which comprises a deterministic component involving the three regression coefficients (

Multiple regression Introduction Multiple regression is a logical extension of the principles of simple linear regression to situations in which there are several predictor variables. For instance if we

### The aspect of the data that we want to describe/measure is the degree of linear relationship between and The statistic r describes/measures the degree

PS 511: Advanced Statistics for Psychological and Behavioral Research 1 Both examine linear (straight line) relationships Correlation works with a pair of scores One score on each of two variables ( and

### 2. Simple Linear Regression

Research methods - II 3 2. Simple Linear Regression Simple linear regression is a technique in parametric statistics that is commonly used for analyzing mean response of a variable Y which changes according

### Null Hypothesis H 0. The null hypothesis (denoted by H 0

Hypothesis test In statistics, a hypothesis is a claim or statement about a property of a population. A hypothesis test (or test of significance) is a standard procedure for testing a claim about a property

### X X X a) perfect linear correlation b) no correlation c) positive correlation (r = 1) (r = 0) (0 < r < 1)

CORRELATION AND REGRESSION / 47 CHAPTER EIGHT CORRELATION AND REGRESSION Correlation and regression are statistical methods that are commonly used in the medical literature to compare two or more variables.

### Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011

Chicago Booth BUSINESS STATISTICS 41000 Final Exam Fall 2011 Name: Section: I pledge my honor that I have not violated the Honor Code Signature: This exam has 34 pages. You have 3 hours to complete this

### STAT 350 Practice Final Exam Solution (Spring 2015)

PART 1: Multiple Choice Questions: 1) A study was conducted to compare five different training programs for improving endurance. Forty subjects were randomly divided into five groups of eight subjects

### " Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

### Basic Statistcs Formula Sheet

Basic Statistcs Formula Sheet Steven W. ydick May 5, 0 This document is only intended to review basic concepts/formulas from an introduction to statistics course. Only mean-based procedures are reviewed,

### Example: Boats and Manatees

Figure 9-6 Example: Boats and Manatees Slide 1 Given the sample data in Table 9-1, find the value of the linear correlation coefficient r, then refer to Table A-6 to determine whether there is a significant

### Univariate Regression

Univariate Regression Correlation and Regression The regression line summarizes the linear relationship between 2 variables Correlation coefficient, r, measures strength of relationship: the closer r is

### NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( )

Chapter 340 Principal Components Regression Introduction is a technique for analyzing multiple regression data that suffer from multicollinearity. When multicollinearity occurs, least squares estimates

### 1) Write the following as an algebraic expression using x as the variable: Triple a number subtracted from the number

1) Write the following as an algebraic expression using x as the variable: Triple a number subtracted from the number A. 3(x - x) B. x 3 x C. 3x - x D. x - 3x 2) Write the following as an algebraic expression

### Module 5: Multiple Regression Analysis

Using Statistical Data Using to Make Statistical Decisions: Data Multiple to Make Regression Decisions Analysis Page 1 Module 5: Multiple Regression Analysis Tom Ilvento, University of Delaware, College

### , has mean A) 0.3. B) the smaller of 0.8 and 0.5. C) 0.15. D) which cannot be determined without knowing the sample results.

BA 275 Review Problems - Week 9 (11/20/06-11/24/06) CD Lessons: 69, 70, 16-20 Textbook: pp. 520-528, 111-124, 133-141 An SRS of size 100 is taken from a population having proportion 0.8 of successes. An

### HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION

HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION HOD 2990 10 November 2010 Lecture Background This is a lightning speed summary of introductory statistical methods for senior undergraduate

### Simple Linear Regression

Chapter Nine Simple Linear Regression Consider the following three scenarios: 1. The CEO of the local Tourism Authority would like to know whether a family s annual expenditure on recreation is related

### 17. SIMPLE LINEAR REGRESSION II

17. SIMPLE LINEAR REGRESSION II The Model In linear regression analysis, we assume that the relationship between X and Y is linear. This does not mean, however, that Y can be perfectly predicted from X.

### Statistiek II. John Nerbonne. March 24, 2010. Information Science, Groningen Slides improved a lot by Harmut Fitz, Groningen!

Information Science, Groningen j.nerbonne@rug.nl Slides improved a lot by Harmut Fitz, Groningen! March 24, 2010 Correlation and regression We often wish to compare two different variables Examples: compare

### Statistics 112 Regression Cheatsheet Section 1B - Ryan Rosario

Statistics 112 Regression Cheatsheet Section 1B - Ryan Rosario I have found that the best way to practice regression is by brute force That is, given nothing but a dataset and your mind, compute everything

### CALCULATIONS & STATISTICS

CALCULATIONS & STATISTICS CALCULATION OF SCORES Conversion of 1-5 scale to 0-100 scores When you look at your report, you will notice that the scores are reported on a 0-100 scale, even though respondents

### Regression, least squares

Regression, least squares Joe Felsenstein Department of Genome Sciences and Department of Biology Regression, least squares p.1/24 Fitting a straight line X Two distinct cases: The X values are chosen

### Notes on Applied Linear Regression

Notes on Applied Linear Regression Jamie DeCoster Department of Social Psychology Free University Amsterdam Van der Boechorststraat 1 1081 BT Amsterdam The Netherlands phone: +31 (0)20 444-8935 email:

### Lesson Lesson Outline Outline

Lesson 15 Linear Regression Lesson 15 Outline Review correlation analysis Dependent and Independent variables Least Squares Regression line Calculating l the slope Calculating the Intercept Residuals and

### Residuals. Residuals = ª Department of ISM, University of Alabama, ST 260, M23 Residuals & Minitab. ^ e i = y i - y i

A continuation of regression analysis Lesson Objectives Continue to build on regression analysis. Learn how residual plots help identify problems with the analysis. M23-1 M23-2 Example 1: continued Case

### Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur. Lecture - 2 Simple Linear Regression

Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur Lecture - 2 Simple Linear Regression Hi, this is my second lecture in module one and on simple

### Chris Slaughter, DrPH. GI Research Conference June 19, 2008

Chris Slaughter, DrPH Assistant Professor, Department of Biostatistics Vanderbilt University School of Medicine GI Research Conference June 19, 2008 Outline 1 2 3 Factors that Impact Power 4 5 6 Conclusions

### Simple Linear Regression in SPSS STAT 314

Simple Linear Regression in SPSS STAT 314 1. Ten Corvettes between 1 and 6 years old were randomly selected from last year s sales records in Virginia Beach, Virginia. The following data were obtained,

### MULTIPLE REGRESSION EXAMPLE

MULTIPLE REGRESSION EXAMPLE For a sample of n = 166 college students, the following variables were measured: Y = height X 1 = mother s height ( momheight ) X 2 = father s height ( dadheight ) X 3 = 1 if

### Bivariate Regression Analysis. The beginning of many types of regression

Bivariate Regression Analysis The beginning of many types of regression TOPICS Beyond Correlation Forecasting Two points to estimate the slope Meeting the BLUE criterion The OLS method Purpose of Regression

### Chapter 10. Key Ideas Correlation, Correlation Coefficient (r),

Chapter 0 Key Ideas Correlation, Correlation Coefficient (r), Section 0-: Overview We have already explored the basics of describing single variable data sets. However, when two quantitative variables

### Technology Step-by-Step Using StatCrunch

Technology Step-by-Step Using StatCrunch Section 1.3 Simple Random Sampling 1. Select Data, highlight Simulate Data, then highlight Discrete Uniform. 2. Fill in the following window with the appropriate

### Statistics II Final Exam - January Use the University stationery to give your answers to the following questions.

Statistics II Final Exam - January 2012 Use the University stationery to give your answers to the following questions. Do not forget to write down your name and class group in each page. Indicate clearly

### Multiple Regression in SPSS STAT 314

Multiple Regression in SPSS STAT 314 I. The accompanying data is on y = profit margin of savings and loan companies in a given year, x 1 = net revenues in that year, and x 2 = number of savings and loan

### Section 1: Simple Linear Regression

Section 1: Simple Linear Regression Carlos M. Carvalho The University of Texas McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/ 1 Regression: General Introduction

### Chapter 7: Simple linear regression Learning Objectives

Chapter 7: Simple linear regression Learning Objectives Reading: Section 7.1 of OpenIntro Statistics Video: Correlation vs. causation, YouTube (2:19) Video: Intro to Linear Regression, YouTube (5:18) -

### 2. What is the general linear model to be used to model linear trend? (Write out the model) = + + + or

Simple and Multiple Regression Analysis Example: Explore the relationships among Month, Adv.\$ and Sales \$: 1. Prepare a scatter plot of these data. The scatter plots for Adv.\$ versus Sales, and Month versus

### Chapter 23. Inferences for Regression

Chapter 23. Inferences for Regression Topics covered in this chapter: Simple Linear Regression Simple Linear Regression Example 23.1: Crying and IQ The Problem: Infants who cry easily may be more easily

### AMS7: WEEK 8. CLASS 1. Correlation Monday May 18th, 2015

AMS7: WEEK 8. CLASS 1 Correlation Monday May 18th, 2015 Type of Data and objectives of the analysis Paired sample data (Bivariate data) Determine whether there is an association between two variables This

### Lecture 18 Linear Regression

Lecture 18 Statistics Unit Andrew Nunekpeku / Charles Jackson Fall 2011 Outline 1 1 Situation - used to model quantitative dependent variable using linear function of quantitative predictor(s). Situation

### Calculate Confidence Intervals Using the TI Graphing Calculator

Calculate Confidence Intervals Using the TI Graphing Calculator Confidence Interval for Population Proportion p Confidence Interval for Population μ (σ is known 1 Select: STAT / TESTS / 1-PropZInt x: number

### 9-3.4 Likelihood ratio test. Neyman-Pearson lemma

9-3.4 Likelihood ratio test Neyman-Pearson lemma 9-1 Hypothesis Testing 9-1.1 Statistical Hypotheses Statistical hypothesis testing and confidence interval estimation of parameters are the fundamental

### Yiming Peng, Department of Statistics. February 12, 2013

Regression Analysis Using JMP Yiming Peng, Department of Statistics February 12, 2013 2 Presentation and Data http://www.lisa.stat.vt.edu Short Courses Regression Analysis Using JMP Download Data to Desktop

### Introduction to Hypothesis Testing. Point estimation and confidence intervals are useful statistical inference procedures.

Introduction to Hypothesis Testing Point estimation and confidence intervals are useful statistical inference procedures. Another type of inference is used frequently used concerns tests of hypotheses.

### 0.1 Estimating and Testing Differences in the Treatment means

0. Estimating and Testing Differences in the Treatment means Is the F-test significant, we only learn that not all population means are the same, but through this test we can not determine where the differences

### Two Related Samples t Test

Two Related Samples t Test In this example 1 students saw five pictures of attractive people and five pictures of unattractive people. For each picture, the students rated the friendliness of the person

### Interaction between quantitative predictors

Interaction between quantitative predictors In a first-order model like the ones we have discussed, the association between E(y) and a predictor x j does not depend on the value of the other predictors

### Inferential Statistics

Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

### Regression Models 1. May 10, 2012 Regression Models

Regression Models 1 May 10, 2012 Regression Models Perhaps the most used statistical methodology in practice is regression analysis. Basically the idea of regression is to relate one variable, called a

### Statistical Modelling in Stata 5: Linear Models

Statistical Modelling in Stata 5: Linear Models Mark Lunt Arthritis Research UK Centre for Excellence in Epidemiology University of Manchester 08/11/2016 Structure This Week What is a linear model? How

### Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

### Inference in Regression Analysis. Dr. Frank Wood

Inference in Regression Analysis Dr. Frank Wood Inference in the Normal Error Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters

### Study Resources For Algebra I. Unit 1C Analyzing Data Sets for Two Quantitative Variables

Study Resources For Algebra I Unit 1C Analyzing Data Sets for Two Quantitative Variables This unit explores linear functions as they apply to data analysis of scatter plots. Information compiled and written

### MTH 140 Statistics Videos

MTH 140 Statistics Videos Chapter 1 Picturing Distributions with Graphs Individuals and Variables Categorical Variables: Pie Charts and Bar Graphs Categorical Variables: Pie Charts and Bar Graphs Quantitative

### An example ANOVA situation. 1-Way ANOVA. Some notation for ANOVA. Are these differences significant? Example (Treating Blisters)

An example ANOVA situation Example (Treating Blisters) 1-Way ANOVA MATH 143 Department of Mathematics and Statistics Calvin College Subjects: 25 patients with blisters Treatments: Treatment A, Treatment

### An analysis appropriate for a quantitative outcome and a single quantitative explanatory. 9.1 The model behind linear regression

Chapter 9 Simple Linear Regression An analysis appropriate for a quantitative outcome and a single quantitative explanatory variable. 9.1 The model behind linear regression When we are examining the relationship

### Chapter 12 : Linear Correlation and Linear Regression

Number of Faculty Chapter 12 : Linear Correlation and Linear Regression Determining whether a linear relationship exists between two quantitative variables, and modeling the relationship with a line, if

### Multiple Linear Regression

Multiple Linear Regression A regression with two or more explanatory variables is called a multiple regression. Rather than modeling the mean response as a straight line, as in simple regression, it is

### Part 2: Analysis of Relationship Between Two Variables

Part 2: Analysis of Relationship Between Two Variables Linear Regression Linear correlation Significance Tests Multiple regression Linear Regression Y = a X + b Dependent Variable Independent Variable

### MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question.

Open book and note Calculator OK Multiple Choice 1 point each MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. Find the mean for the given sample data.