Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur. Lecture - 2 Simple Linear Regression

Save this PDF as:

Size: px
Start display at page:

Download "Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur. Lecture - 2 Simple Linear Regression"

Transcription

1 Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur Lecture - 2 Simple Linear Regression Hi, this is my second lecture in module one and on simple linear regression. In the first lecture, we have introduced simple linear regression model and they have learnt how to estimate the regression coefficients using least square technique. (Refer Slide Time: 01:25) Here is the content of today s lecture first will give one example on simple linear regression. And then we talk about useful properties of least square fit and then the statistical property of least square estimator.

2 (Refer Slide Time: 01:53) Well let me just recall the simple linear regression model. The general form of simple linear regression model is a Y equal to beta naught plus beta 1 X plus epsilon. (Refer Slide Time: 02:08) Were Y is the response variable X is the regressor variable and epsilon is the error term and beta naught is intercept and beta 1 is slope and we call beta naught into beta 1there regression coefficients. Now, given a set of observations say X i Y i for i equal to 1 to n we learned how to estimate the regression coefficients beta naught and beta 1 using least square technique. So, least squares method determines the parameters means the

3 regression coefficients beta naught and beta 1 by minimizing residual sum of squares SS residual. Which is equal to e i square i equal to 1 to n, which is basically equal to the difference between the observed response value and the estimated response value y i hat. So, this is the residual and y i hat this is y i minus t i hat is the is equal to beta naught hat plus beta 1 hat X. So, this is beta naught hat minus beta 1 hat X i whole square. Because the fitted equation is y i hat equal to beta naught hat plus beta 1 hat X i. So, we learned that beta naught I mean this SS residual or residual sum of squares is minimum. When beta naught hat is equal to Y bar minus beta 1 hat X bar. (Refer Slide Time: 05:53) And beta 1 hat is equal to summation x i minus x bar y i minus y bar by summation x i minus x bar whole square. Well this quantity is also denoted by the symbol S x y by S x x and this one can also be written in the form summation x i y i minus x bar y bar n times by summation x i square minus n into x bar square. This is not difficult to observe just a simple algebra shows that this quantity is equal to this quantity. So, let us move to the disney toy example this is the cost on advertising and this is the sales amount, so these are the x i values and this are the y i values. And even to fit a straight-line model to this data this table shows.

4 (Refer Slide Time: 08:02) Summation X i is equal to 15 summation, Y i is equal to 10 summation, X i square is equal to 55 summation, Y i square is equal to 26 and summation X i Y i is equal to 37. (Refer Slide Time: 08:31) And then we can compute the value of beta 1 hat. So, beta 1 hat is equal to summation X i Y i minus n times X bar into Y bar by summation X i square minus n times X bar square which is equal to 0.7. And similarly beta naught hat is equal to Y bar minus beta 1 hat X bar which is equal to minus 0.1. Well so the fitted equation is Y hat equal to beta naught which is equal to minus 0.1 plus 0.7 into X. So, this is the fitted equation for the

5 given observations, now what is the interpretation of this regression coefficient beta 1. So, it says that that the expected value of the response variable or. (Refer Slide Time: 10:03) The expected sales amount is increased by 0.7 units for each 1 dollar increase in advertising and the interpretation of the coefficient beta naught hat is that. (Refer Slide Time: 10:17) What is the average sales amount when X is equal to 0 beta naught is equal to Y hat, when x is equal to 0. So, beta naught beta naught hat this basically gives some idea about the average value of sales amount when the advertising cost is equal to 0. So I mean its

6 very difficult explain why it is so because see we can expect some sales amount without advertising also. But here it is negative anyway, so the beta naught hat is the average sales volume which is equal to minus 0.1 when the advertising is equal to 0. (Refer Slide Time: 11:24) Well next we move to the useful properties of the least square fit. It says that we know what is residual is e i is the difference between the observed response value and the predicted or the estimated response value. So, it says that the sum of residuals in any regression model that contains a intercept beta naught is always 0. I am going to prove this one, so this summation e i is equal to 0. And the second property says that the sum of the observed value is equal to the sum of the fitted values y i hat. So I mean this second property is consequence of the first property it is obtained from this one only. Let me prove the first property which says that the some of residuals in any regression model that contains an intercept beta naught is always equal to 0. So, what it to do is that we have to recall the.

7 (Refer Slide Time: 13:00) L S method least squares method least squares method determines the parameter beta naught and beta 1 by minimizing by minimizing SS residual that this residual sum of square. Which is equal to summation e i square and this is basically equal to summation y i minus y i hat square. And this is equal to summation y i minus beta naught hat minus beta 1 hat x i square well. So, least squares method or L S method determines the parameter beta naught hat and beta 1 hat by minimizing this quantity. So, what we do is that we differentiate the residual sum of square with respect to beta naught hat and we equate that with 0. We did the first normally equations the normally equations differentiate SS residual with beta naught hat that gives one normally equation. And again we differentiate residual sum of squares with respect to beta 1 hat that gives another normally equation the first normally equation. We differentiate SS residual with respect to 0, sorry with respect to beta naught hat that gives summation y i minus beta naught hat minus beta 1 hat x i 2 times of this and one negative sign here. So, and we equate this 2 equal to 0 this is the first normally equation and which is nothing but see. This is nothing but y i minus y i hat, so this one is nothing but summation e i equal to 0. To the first normally equation says that the sum of the residual is equal to 0. And this residual sum of residual e i is nothing but summation. So, this is the first property the second property says that well this is residual some of residual is equal to y i minus y i

8 hat this is the residual ith residual. And we know this is equal to 0 from the first property, which implies that summation y i is equal to summation y i hat. So, this one is the consequence of the previous property or the first property it says that the sum of observed values equal to is equal to the sum of the fitted value well. So, next will move to the third property. (Refer Slide Time: 16:52) It says that summation x i e i equal to 0, that means sum of residuals waited by the corresponding value of the regressor variable is equal to 0. So, sum of residual e i waited by the corresponding value of the regression variable this is the value of the ith regression variable this is equal to 0. And the fourth property says that summation y i hat e i is equal to 0. That means the sum of residuals waited by the corresponding fitted value of the response variable is equal to 0, well let me prove this two properties. Property three and property four well by differentiating this residual sum of square with respect to beta naught hat. We got this normally equation this is the first normally equation, now again we differentiate this normally equation sorry this some of residual with respect to beta 1 hat.

9 (Refer Slide Time: 18:28) And that gives summation y i minus beta naught hat minus beta 1 hat x i and we are differentiating with respect to beta 1 hat. So, this quantity will be multiplied by minus 2 into x i, so minus 2 into x i which is equal to 0. This is the second normal equation and this one is nothing but summation y i. And this part is y i hat into x i equal to 0, which implies summation e i x i equal to 0 which is the third property. Now, the fourth property is summation e i y i hat this is equal to 0. But, summation e i y i is not equal to 0 you should note that how to prove that summation e i y i hat equal to 0. This is again consequence of the first property and the third property well this can be written as summation e i what is a y i hat y i hat is equal to beta naught hat plus beta 1 hat x i which is equal to 0. Which is equal to which is equal to summation beta naught hat e i plus beta 1 hat summation e i x i. Now, see summation e i from the first property this is equal to 0. So, this is equal to 0 plus the third property says that summation e i x i equal to 0. So, this is basically beta naught hat into 0 plus beta 1 hat into 0. So, this is equal to 0, so you prove that summation e i y i hat equal to 0. Ok, so these are the some properties of least square fit and will be using them in the future.

10 (Refer Slide Time: 21:30) Next move to the statistical properties of least square estimators. So, we have estimated the regression coefficient beta naught and beta 1 using least square estimators. And we are going to prove that both beta naught hat and beta 1 hat are unbiased estimator of beta naught an beta 1 respectively well let me prove that. (Refer Slide Time: 22:20) First I will prove that beta naught hat and beta 1 hat are linear combinations of observations y i, that means we want to say that they are linear estimator. Ok for example, considered beta 1 hat, so what is beta 1 hat beta 1 hat is equal to summation x i

11 minus x bar into y i minus y bar by summation x i minus x bar whole square. Now, I said that this is linear combination of the response variables y i s other observation y i s. This can be written as this can be written as summation x i minus x bar into y i only by summation x i minus x bar whole square this is it i mean this two quantity are same. One can very easily prove that summation x i minus x bar into y bar that quantity is equal to 0. This is not difficult to prove that now this is from i equal to 1 to n and i equal to 1 to n. Now, this can be written as summation C i y i were C i is equal to here i equal to 1 to n were C i is equal to x i minus x bar by summation x i minus x bar whole square. So, I prove that beta 1 hat is a linear combination of the observations y i. Similarly, one can prove that beta naught hat is also easily what is beta naught hat beta naught hat. (Refer Slide Time: 25:38) Beta naught hat is equal to y bar minus beta 1 hat x bar. So, this is also a linear combination of the observation. Because this is nothing but this is nothing but summation y i from i equal to 1 to n 1 to n and minus beta 1 hat x bar. We already prove that beta 1 hat is a linear combination of the observation y i and the first term is also a linear combination of y i. So, the whole thing is a linear combinations of the observations y i. So, this just prove that the estimator we got they are linear estimator they are linear in y i next let me prove that the estimator beta naught and beta 1 they are.

12 (Refer Slide Time: 26:55) Unbiased estimator that is we are going to prove that expectation of beta 1 hat is equal to beta 1. So, if this is true then we call beta 1 is an unbiased estimator. Well let me start from here the simple linear regression model is y i is equal to beta naught plus beta 1 x i plus epsilon i. Now, y bar y bar which is equal to summation y i i equal to 1 to n. This is equal to beta naught plus beta 1 x i will be replaced by x bar and epsilon i will be replaced by epsilon bar. So, were x bar were x bar is of course, equal to summation x i 1 to n and epsilon bar is also equal to summation epsilon y summation epsilon y 1 to n. Then from here y i minus y bar is equal to y i minus y bar is equal to beta 1 into x i minus x bar plus epsilon i minus epsilon bar. So, the expected value of the expected value of y i minus y bar is equal to. See one thing you should observe that you should always remember that y is a random variable. The response variable y is a random variable, but x is not a random variable it is its controlled variable. So, for given i this is just a constant, so expected value of y i minus y bar is equal to beta 1 x i minus x bar plus expectation of epsilon i minus epsilon bar. Now, epsilon i is the random error which follows which is a random variable and this follows we are assumed that epsilon i follows normal 0 sigma square. So, the expected value of e i is equal to 0 and similarly the expected value of epsilon bar is also equal to 0. The expected value of y i minus y bar this term is going to be 0 is equal to beta 1 into x i minus x bar. Now, our aim is to prove that beta 1 hat is an unbiased estimated of beta 1,

13 so expectation of beta 1 hat is equal to expectation of what is beta 1 hat beta 1 hat is equal to summation x i minus x bar y i minus y bar by summation x i minus x bar whole square. So, this one is nothing but summation x i minus x bar into expectation of y i minus y bar by summation x i minus x bar whole square. (Refer Slide Time: 32:37) Now note that we proved that expectation of y i minus y bar is equal to beta 1 into x i minus x bar, so expectation of beta 1 hat is equal to summation x i minus x bar. (Refer Slide Time: 33:04)

14 And this quantity is equal to beta 1 beta 1 x i minus x bar by summation x i minus x bar whole square. So, this is nothing but beta 1 only because this one is nothing but x i minus x bar whole square. So, he proved that beta 1 is an beta 1 hat is an unbiased estimated of beta 1 similarly, next we prove that beta 1 hat. (Refer Slide Time: 33:53) Sorry beta naught hat is also unbiased that means we are going to prove that expectation expected value of beta naught hat is equal to beta naught, this is equal to expectation of y bar minus beta 1 hat x bar. So, what is y bar y bar equal to we proved in the previous light y bar is nothing but beta naught plus beta 1 x bar minus beta 1 hat x bar. So, this is equal to expectation of beta naught is beta naught expectation of beta 1 x bar is equal to beta 1 x bar. Now, the expectation of beta 1 hat just now we proved that expectation of beta naught beta 1 hat is equal to beta 1 hat sorry beta 1. So, this is equal to beta 1 x bar which is equal to beta naught, so we prove that both beta 1 hat and beta naught hat are unbiased.

15 (Refer Slide Time: 36:01) Next, will talk about the variance of the variance of beta 1 hat and beta naught hat the variance of beta 1 hat is equal to sigma square by S x x and variance of beta naught hat is equal to sigma square into 1 by n plus x bar square by S x x. Anyway I mean we need to know how to how to derive these things well. (Refer Slide Time: 36:36) So, next we talk about the variance of variance of beta naught hat and beta 1 hat. So, the variance of beta 1 hat is equal to variance of what is beta 1 hat beta 1 hat is equal to summation x i minus x bar into y i minus y bar by summation x i minus x bar whole

16 square. Well now this can be written as the variance of summation x i minus x bar into y i by summation x i minus x bar whole square. And this is nothing but see I before also I proved that this estimator is a linear combination of the observation y i. So, the variance of this quantity is and since y i y i is are independent you know well this is equal to variance of summation C i y i, were C i were C i is equal to x i minus x bar by summation x I minus x bar. Now, we know that the variance of the variance of summation C i y i is equal to summation sum of variances C i square variance of y i. This is true because because y i s are y i s are independent and we know that variance of y i is equal to sigma square this is equal to summation c i square sigma square. Now, what is summation C i square summation C i square we know this is this is C i. So, variance of beta 1 hat is equal to sigma square into summation C i square which is equal to summation x i minus x bar C i square. (Refer Slide Time: 40:36) So, summation x i minus x bar whole square by summation x i minus x bar whole square and the square of the whole thing. So, just replacing C i square by its value and this becomes sigma square by summation x i minus x bar whole square, which is equal to sigma square by S x x by notation. So, he proved that variance of beta 1 hat is equal to sigma square by S x x.

17 (Refer Slide Time: 41:54) So, next we talk about the variance of beta naught hat variance of beta naught hat is equal to variance of y bar minus beta 1 hat x bar right. Now, this variance can be written as variance of y bar plus the variance of beta 1 hat x bar minus 2 times x bar the covariance of y bar and beta 1 hat right. Now, you know the variance of y bar i hope you know it it is equal to its not difficult to prove that this is equal to sigma square by n. Let me prove this variance of y bar is equal to the variance of 1 by n summation y i know y i is an independent. That is assumption we made at the beginning because e i is an independent. So, epsilon i s are independent y i s are also an independent the variance of this quantity is equal to summation of variance of y i by n square right and variance of y i is equal to sigma square. So, summation sigma square n times 1 to n y n square, so this is basically n sigma square by n square. This is equal to sigma square by n the variance of y bar is equal to sigma square by n, we know the variance of beta 1 hat and this is the constant quantity. So, the variance of this quantity is x bar square into the variance of beta 1 hat which is equal to which is equal to sigma square by S x x just we proved. Now, what about this covariance this is going to be this co-variance is going to be 0 this is 2 x bar into 0 into 0, but we need to prove this one. Ok the covariance between this can be proved that the covariance is equal to 0.

18 (Refer Slide Time: 45:07) The covariance between y bar and beta 1 hat is equal to the covariance between what is y bar y bar is equal to summation y i by n. And beta 1 hat is equal to summation x i minus x bar into y i by summation x i minus x bar whole square right. Now, this covariance is see y i are independent, so the covariance between y i and y j is equal to 0 when i is not equal to j. So, this is nothing but the summation of x i minus x bar the covariance between y i and y i. So, which is nothing but the variance of y i right by summation x i minus x bar whole square and 1 n here. This is equal to this is equal to summation x i minus x bar variance of y i is equal to sigma square by n into summation x i minus x bar whole square. And this quantity see summation x i minus x bar is equal to 0. That is why the numerator is 0, so this is equal to sigma square into summation x i minus x bar this quantity is 0 always by n into summation x i minus x bar whole square. So, this is nothing this is equal to 0, because of the fact that summation x i minus x bar is always equal to 0. So, he proved that so he proved that covariance is equal to 0.

19 (Refer Slide Time: 47:53) That means the variance of beta naught hat is equal to sigma square by n plus x bar square into variance of beta 1 hat which is equal to sigma square by S x x which we just proved. This is sigma square by into 1 by n plus x bar square by S x x, so we found. So, we proved that both the beta naught and beta 1 they are unbiased estimated sigma square. (Refer Slide Time: 48:48) And also we proved that the variance of beta naught hat is equal to sigma square 1 by n plus x bar square by S x x and the variance of beta 1 hat is equal to sigma square by S x x. Now, see both the variance formula the variance for beta naught hat and beta 1 hat

20 both the involved a sigma square. So, but we do not know what is the value of sigma square, so sigma square must be replaced by its estimators. So, you need to estimate the value of sigma square, so next we talk about the estimation of sigma square, how to how to estimate sigma square. (Refer Slide Time: 50:06) Estimation of sigma square well the estimation of sigma square is obtained from the residual sum of square SS residual. And finally will be proving that expected value of SS residual by n minus 2. This thing is equal to sigma square that means the sum of residual sorry residual sum of square by n minus 2 is unbiased estimated of sigma square. And will be we can compute the value of residual sum of square given a set of observations and we know n. So, using this formula we can we can estimate the value of the population variance sigma square. So, well we need to prove this one SS residual is equal to summation e i square. That we know and which is equal to the difference the ith residual is the difference between ith observation and the estimated value y i at. So, this is equal to this is equal to square here this is equal to summation y i minus what is y i at is equal to beta naught hat minus beta 1 hat x i whole square. So, what we do here is that we know beta naught hat his equal to just trying to find a convenient form for SS residual. So, beta naught hat is equal to y bar minus beta 1 hat x bar right. So, will just plug this value here this is equal to summation y i minus y bar minus beta 1 hat x i plus beta 1 hat x bar whole square.

21 This is equal to summation y i minus y bar minus beta 1 hat x i minus x bar whole square, now this can be written as summation y i minus y bar whole square minus. Let me write plus beta 1 hat square summation x i minus x bar whole square minus 2 times beta 1 hat summation x i minus x bar into y i minus y bar right. (Refer Slide Time: 54:23) So, this can be written in the form S y y plus beta 1 hat square is x x minus 2 beta 1 hat is x y, just notation. (Refer Slide Time: 54:55)

22 Now, see we know that beta 1 hat is equal to S x y by S x x right so what I will do is that I will replace this S x S x y by beta 1 hat into S x x. So, this is equal to S y y minus beta 1 hat is S x x minus 2 beta 1 hat. Now, replace this one by beta 1 hat S x x into this becomes S y y plus beta 1 hat S x x minus two times. This is square here square here beta 1 hat square S x x, so which is equal to S x sorry S y y minus beta 1 hat S x x. This is a convenient form of SS residual and we are going to use this one in the next class. To prove that because you need to find the expected value of this one to prove. (Refer Slide Time: 56:45) The expectation of SS to prove that expectation of SS residual by n minus 2 equal to sigma square well, so will continue in the in the next class. Thank you.

Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur

Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur Lecture - 7 Multiple Linear Regression (Contd.) This is my second lecture on Multiple Linear Regression

Advanced Engineering Mathematics Prof. Jitendra Kumar Department of Mathematics Indian Institute of Technology, Kharagpur

Advanced Engineering Mathematics Prof. Jitendra Kumar Department of Mathematics Indian Institute of Technology, Kharagpur Lecture No. # 28 Fourier Series (Contd.) Welcome back to the lecture on Fourier

Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras. Lecture - 28

Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras Lecture - 28 In the earlier lectures, we have seen formulation for 3 node linear triangular

Lecture - 32 Regression Modelling Using SPSS

Applied Multivariate Statistical Modelling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur Lecture - 32 Regression Modelling Using SPSS (Refer

Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras. Lecture - 01

Finite Element Analysis Prof. Dr. B. N. Rao Department of Civil Engineering Indian Institute of Technology, Madras Lecture - 01 Welcome to the series of lectures, on finite element analysis. Before I start,

Advanced 3G and 4G Wireless Communication Prof. Aditya K. Jagannatham Department of Electrical Engineering Indian Institute of Technology, Kanpur

Advanced 3G and 4G Wireless Communication Prof. Aditya K. Jagannatham Department of Electrical Engineering Indian Institute of Technology, Kanpur Lecture - 3 Rayleigh Fading and BER of Wired Communication

A Basic Course in Real Analysis Prof. P. D. Srivastava Department of Mathematics Indian Institute of Technology, Kharagpur

A Basic Course in Real Analysis Prof. P. D. Srivastava Department of Mathematics Indian Institute of Technology, Kharagpur Lecture - 3 Continuum and Exercises So the last lecture we have discussed the

Power System Analysis Prof. A. K. Sinha Department of Electrical Engineering Indian Institute of Technology, Kharagpur. Lecture - 19 Power Flow IV

Power System Analysis Prof. A. K. Sinha Department of Electrical Engineering Indian Institute of Technology, Kharagpur Lecture - 19 Power Flow IV Welcome to lesson 19 on Power System Analysis. In this

Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur

Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur Module No. #01 Lecture No. #15 Special Distributions-VI Today, I am going to introduce

Section 14 Simple Linear Regression: Introduction to Least Squares Regression

Slide 1 Section 14 Simple Linear Regression: Introduction to Least Squares Regression There are several different measures of statistical association used for understanding the quantitative relationship

Lecture No. # 02 Prologue-Part 2

Advanced Matrix Theory and Linear Algebra for Engineers Prof. R.Vittal Rao Center for Electronics Design and Technology Indian Institute of Science, Bangalore Lecture No. # 02 Prologue-Part 2 In the last

Simple Regression Theory II 2010 Samuel L. Baker

SIMPLE REGRESSION THEORY II 1 Simple Regression Theory II 2010 Samuel L. Baker Assessing how good the regression equation is likely to be Assignment 1A gets into drawing inferences about how close the

17. SIMPLE LINEAR REGRESSION II

17. SIMPLE LINEAR REGRESSION II The Model In linear regression analysis, we assume that the relationship between X and Y is linear. This does not mean, however, that Y can be perfectly predicted from X.

Lecture 5: Linear least-squares Regression III: Advanced Methods William G. Jacoby Department of Political Science Michigan State University http://polisci.msu.edu/jacoby/icpsr/regress3 Simple Linear Regression

Basic Quantum Mechanics Prof. Ajoy Ghatak Department of Physics. Indian Institute of Technology, Delhi

Basic Quantum Mechanics Prof. Ajoy Ghatak Department of Physics. Indian Institute of Technology, Delhi Module No. # 02 Simple Solutions of the 1 Dimensional Schrodinger Equation Lecture No. # 7. The Free

Advanced Hydraulics Prof. Dr. Suresh. A. Kartha Department of Civil Engineering Indian Institute of Technology, Guwahati

Advanced Hydraulics Prof. Dr. Suresh. A. Kartha Department of Civil Engineering Indian Institute of Technology, Guwahati Module - 1 Open Channel Flow Lecture -6 Specific Energy and Critical Flow Very good

Simple Linear Regression Chapter 11

Simple Linear Regression Chapter 11 Rationale Frequently decision-making situations require modeling of relationships among business variables. For instance, the amount of sale of a product may be related

Simple Regression Theory I 2010 Samuel L. Baker

SIMPLE REGRESSION THEORY I 1 Simple Regression Theory I 2010 Samuel L. Baker Regression analysis lets you use data to explain and predict. A simple regression line drawn through data points In Assignment

Notes 5: More on regression and residuals ECO 231W - Undergraduate Econometrics

Notes 5: More on regression and residuals ECO 231W - Undergraduate Econometrics Prof. Carolina Caetano 1 Regression Method Let s review the method to calculate the regression line: 1. Find the point of

The Classical Linear Regression Model

The Classical Linear Regression Model 1 September 2004 A. A brief review of some basic concepts associated with vector random variables Let y denote an n x l vector of random variables, i.e., y = (y 1,

17.0 Linear Regression

17.0 Linear Regression 1 Answer Questions Lines Correlation Regression 17.1 Lines The algebraic equation for a line is Y = β 0 + β 1 X 2 The use of coordinate axes to show functional relationships was

1.6 The Order of Operations

1.6 The Order of Operations Contents: Operations Grouping Symbols The Order of Operations Exponents and Negative Numbers Negative Square Roots Square Root of a Negative Number Order of Operations and Negative

(Refer Slide Time 1.50)

Discrete Mathematical Structures Dr. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Module -2 Lecture #11 Induction Today we shall consider proof

e = random error, assumed to be normally distributed with mean 0 and standard deviation σ

1 Linear Regression 1.1 Simple Linear Regression Model The linear regression model is applied if we want to model a numeric response variable and its dependency on at least one numeric factor variable.

BASIC ELECTRONICS PROF. T.S. NATARAJAN DEPT OF PHYSICS IIT MADRAS LECTURE-4 SOME USEFUL LAWS IN BASIC ELECTRONICS

BASIC ELECTRONICS PROF. T.S. NATARAJAN DEPT OF PHYSICS IIT MADRAS LECTURE-4 SOME USEFUL LAWS IN BASIC ELECTRONICS Hello everybody! In a series of lecture on basic electronics, learning by doing, we now

Section 1.9 Algebraic Expressions: The Distributive Property

Section 1.9 Algebraic Expressions: The Distributive Property Objectives In this section, you will learn to: To successfully complete this section, you need to understand: Apply the Distributive Property.

Cost-Volume-Profit Analysis

Cost-Volume-Profit Analysis Cost-volume-profit (CVP) analysis is used to determine how changes in costs and volume affect a company's operating income and net income. In performing this analysis, there

Materials and Energy Balance in Metallurgical Processes. Prof. S.C. Koria. Department of Materials Science and Engineering

Materials and Energy Balance in Metallurgical Processes Prof. S.C. Koria Department of Materials Science and Engineering Indian Institute of Technology, Kanpur Module No. # 01 Lecture No. # 06 Thermochemistry

Foundation Engineering Prof. Mahendra Singh Department of Civil Engineering Indian Institute of Technology, Roorkee

Foundation Engineering Prof. Mahendra Singh Department of Civil Engineering Indian Institute of Technology, Roorkee Module - 03 Lecture - 09 Stability of Slopes Welcome back to the classes of on this Stability

Digital System Design Prof. D. Roychoudhury Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur

Digital System Design Prof. D. Roychoudhury Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture - 03 Digital Logic - I Today, we will start discussing on Digital

Linear Regression with One Regressor

Linear Regression with One Regressor Michael Ash Lecture 8 Linear Regression with One Regressor a.k.a. Bivariate Regression An important distinction 1. The Model Effect of a one-unit change in X on the

Algebraic expressions are a combination of numbers and variables. Here are examples of some basic algebraic expressions.

Page 1 of 13 Review of Linear Expressions and Equations Skills involving linear equations can be divided into the following groups: Simplifying algebraic expressions. Linear expressions. Solving linear

Processing of Semiconducting Materials Prof. Pallab Banerji Material Science Centre Indian Institute of Technology, Kharagpur

Processing of Semiconducting Materials Prof. Pallab Banerji Material Science Centre Indian Institute of Technology, Kharagpur Lecture - 27 Characterization II Let us define the following parameters for

Managerial Economics Prof. Trupti Mishra S.J.M School of Management Indian Institute of Technology, Bombay. Lecture 9 Theory of Demand (Contd.

Managerial Economics Prof. Trupti Mishra S.J.M School of Management Indian Institute of Technology, Bombay Lecture 9 Theory of Demand (Contd.) Welcome to the second session of module two; module two talks

Statistical Foundations: Measures of Location and Central Tendency and Summation and Expectation

Statistical Foundations: and Central Tendency and and Lecture 4 September 5, 2006 Psychology 790 Lecture #4-9/05/2006 Slide 1 of 26 Today s Lecture Today s Lecture Where this Fits central tendency/location

(Refer Slide Time: 2:03)

Control Engineering Prof. Madan Gopal Department of Electrical Engineering Indian Institute of Technology, Delhi Lecture - 11 Models of Industrial Control Devices and Systems (Contd.) Last time we were

Procedure In each case, draw and extend the given series to the fifth generation, then complete the following tasks:

Math IV Nonlinear Algebra 1.2 Growth & Decay Investigation 1.2 B: Nonlinear Growth Introduction The previous investigation introduced you to a pattern of nonlinear growth, as found in the areas of a series

1. The Classical Linear Regression Model: The Bivariate Case

Business School, Brunel University MSc. EC5501/5509 Modelling Financial Decisions and Markets/Introduction to Quantitative Methods Prof. Menelaos Karanasos (Room SS69, Tel. 018956584) Lecture Notes 3 1.

A Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution

A Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 4: September

Regression, least squares

Regression, least squares Joe Felsenstein Department of Genome Sciences and Department of Biology Regression, least squares p.1/24 Fitting a straight line X Two distinct cases: The X values are chosen

Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression

Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression Objectives: To perform a hypothesis test concerning the slope of a least squares line To recognize that testing for a

Part 1 Expressions, Equations, and Inequalities: Simplifying and Solving

Section 7 Algebraic Manipulations and Solving Part 1 Expressions, Equations, and Inequalities: Simplifying and Solving Before launching into the mathematics, let s take a moment to talk about the words

Fundamentals of Operations Research Prof.G. Srinivasan Department of Management Studies Lecture No. # 03 Indian Institute of Technology, Madras

Fundamentals of Operations Research Prof.G. Srinivasan Department of Management Studies Lecture No. # 03 Indian Institute of Technology, Madras Linear Programming solutions - Graphical and Algebraic Methods

Managerial Economics Prof. Trupti Mishra S.J.M. School of Management Indian Institute of Technology, Bombay. Lecture - 13 Consumer Behaviour (Contd )

(Refer Slide Time: 00:28) Managerial Economics Prof. Trupti Mishra S.J.M. School of Management Indian Institute of Technology, Bombay Lecture - 13 Consumer Behaviour (Contd ) We will continue our discussion

Lecture 5: Correlation and Linear Regression

Lecture 5: Correlation and Linear Regression 3.5. (Pearson) correlation coefficient The correlation coefficient measures the strength of the linear relationship between two variables. The correlation is

Chapter 11: Two Variable Regression Analysis

Department of Mathematics Izmir University of Economics Week 14-15 2014-2015 In this chapter, we will focus on linear models and extend our analysis to relationships between variables, the definitions

2.3 Subtraction of Integers

2.3 Subtraction of Integers The key to subtraction of real numbers is to forget what you might have learned in elementary school about taking away. With positive numbers you can reasonably say take 3 away

The Simple Linear Regression Model: Specification and Estimation

Chapter 3 The Simple Linear Regression Model: Specification and Estimation 3.1 An Economic Model Suppose that we are interested in studying the relationship between household income and expenditure on

Sigma notation. Jackie Nicholas

Mathematics Learning Centre Sigma notation Jackie Nicholas c 2005 University of Sydney Mathematics Learning Centre, University of Sydney 1 1 Sigma Notation 1.1 Understanding Sigma Notation The symbol Σ

db, dbm, dbw db 10 log (x) where x is unitless! For example, amplifier gain is a unitless value!

2/15/2005 db.doc 1/9 db, dbm, dbw Decibel (db), is a specific function that operates on a unitless parameter: db log (x) where x is unitless! Q: A unitless parameter! What good is that!? A: Many values

Mathematical Notation and Symbols

Mathematical Notation and Symbols 1.1 Introduction This introductory block reminds you of important notations and conventions used throughout engineering mathematics. We discuss the arithmetic of numbers,

(Refer Slide Time: 00:00:56 min)

Numerical Methods and Computation Prof. S.R.K. Iyengar Department of Mathematics Indian Institute of Technology, Delhi Lecture No # 3 Solution of Nonlinear Algebraic Equations (Continued) (Refer Slide

Relationship of two variables

Relationship of two variables A correlation exists between two variables when the values of one are somehow associated with the values of the other in some way. Scatter Plot (or Scatter Diagram) A plot

2. Linear regression with multiple regressors

2. Linear regression with multiple regressors Aim of this section: Introduction of the multiple regression model OLS estimation in multiple regression Measures-of-fit in multiple regression Assumptions

Module 3: Correlation and Covariance

Using Statistical Data to Make Decisions Module 3: Correlation and Covariance Tom Ilvento Dr. Mugdim Pašiƒ University of Delaware Sarajevo Graduate School of Business O ften our interest in data analysis

Univariate Regression

Univariate Regression Correlation and Regression The regression line summarizes the linear relationship between 2 variables Correlation coefficient, r, measures strength of relationship: the closer r is

where b is the slope of the line and a is the intercept i.e. where the line cuts the y axis.

Least Squares Introduction We have mentioned that one should not always conclude that because two variables are correlated that one variable is causing the other to behave a certain way. However, sometimes

Computer Graphics Prof. Sukhendu Das Dept. of Computer Science and Engineering Indian Institute of Technology, Madras Lecture 7 Transformations in 2-D

Computer Graphics Prof. Sukhendu Das Dept. of Computer Science and Engineering Indian Institute of Technology, Madras Lecture 7 Transformations in 2-D Welcome everybody. We continue the discussion on 2D

1) Write the following as an algebraic expression using x as the variable: Triple a number subtracted from the number

1) Write the following as an algebraic expression using x as the variable: Triple a number subtracted from the number A. 3(x - x) B. x 3 x C. 3x - x D. x - 3x 2) Write the following as an algebraic expression

Partial Fractions. Combining fractions over a common denominator is a familiar operation from algebra:

Partial Fractions Combining fractions over a common denominator is a familiar operation from algebra: From the standpoint of integration, the left side of Equation 1 would be much easier to work with than

(Refer Slide Time: 1:41)

Discrete Mathematical Structures Dr. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture # 10 Sets Today we shall learn about sets. You must

Math 018 Review Sheet v.3

Math 018 Review Sheet v.3 Tyrone Crisp Spring 007 1.1 - Slopes and Equations of Lines Slopes: Find slopes of lines using the slope formula m y y 1 x x 1. Positive slope the line slopes up to the right.

2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

Chapter 8. Linear Regression. Copyright 2012, 2008, 2005 Pearson Education, Inc.

Chapter 8 Linear Regression Copyright 2012, 2008, 2005 Pearson Education, Inc. Fat Versus Protein: An Example The following is a scatterplot of total fat versus protein for 30 items on the Burger King

" Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

Managerial Economics Prof. Trupti Mishra S.J.M. School of Management Indian Institute of Technology, Bombay

(Refer Slide Time: 00:39) Managerial Economics Prof. Trupti Mishra S.J.M. School of Management Indian Institute of Technology, Bombay Lecture - 3 Introduction to Managerial Economics (Contd ) So, welcome

Example: Boats and Manatees

Figure 9-6 Example: Boats and Manatees Slide 1 Given the sample data in Table 9-1, find the value of the linear correlation coefficient r, then refer to Table A-6 to determine whether there is a significant

Sample Problems. Lecture Notes Equations with Parameters page 1

Lecture Notes Equations with Parameters page Sample Problems. In each of the parametric equations given, nd the value of the parameter m so that the equation has exactly one real solution. a) x + mx m

Outline. Topic 4 - Analysis of Variance Approach to Regression. Partitioning Sums of Squares. Total Sum of Squares. Partitioning sums of squares

Topic 4 - Analysis of Variance Approach to Regression Outline Partitioning sums of squares Degrees of freedom Expected mean squares General linear test - Fall 2013 R 2 and the coefficient of correlation

12-1 Multiple Linear Regression Models

12-1.1 Introduction Many applications of regression analysis involve situations in which there are more than one regressor variable. A regression model that contains more than one regressor variable is

ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems

Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed

12/31/2016. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2

PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 Understand linear regression with a single predictor Understand how we assess the fit of a regression model Total Sum of Squares

Inference in Regression Analysis. Dr. Frank Wood

Inference in Regression Analysis Dr. Frank Wood Inference in the Normal Error Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters

The basic unit in matrix algebra is a matrix, generally expressed as: a 11 a 12. a 13 A = a 21 a 22 a 23

(copyright by Scott M Lynch, February 2003) Brief Matrix Algebra Review (Soc 504) Matrix algebra is a form of mathematics that allows compact notation for, and mathematical manipulation of, high-dimensional

SPSS Guide: Regression Analysis

SPSS Guide: Regression Analysis I put this together to give you a step-by-step guide for replicating what we did in the computer lab. It should help you run the tests we covered. The best way to get familiar

Quadratic Modeling Business 10 Profits In this activity, we are going to look at modeling business profits. We will allow q to represent the number of items manufactured and assume that all items that

The aspect of the data that we want to describe/measure is the degree of linear relationship between and The statistic r describes/measures the degree

PS 511: Advanced Statistics for Psychological and Behavioral Research 1 Both examine linear (straight line) relationships Correlation works with a pair of scores One score on each of two variables ( and

Digital System Design Prof. D Roychoudhry Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur

Digital System Design Prof. D Roychoudhry Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture - 04 Digital Logic II May, I before starting the today s lecture

Managerial Accounting Prof. Dr. Varadraj Bapat School of Management Indian Institute of Technology, Bombay

Managerial Accounting Prof. Dr. Varadraj Bapat School of Management Indian Institute of Technology, Bombay Lecture - 33 Standard Costing - Mix, Yield and Fixed Overhead Variances Dear students, in the

Advanced Hydraulics Prof. Dr. Suresh A Kartha Department of Civil Engineering Indian Institute of Technology, Guwahati

Advanced Hydraulics Prof. Dr. Suresh A Kartha Department of Civil Engineering Indian Institute of Technology, Guwahati Module - 1 Open Channel Flow Lecture - 9 Critical Flow Computations Part-2 (Refer

Process Control and Instrumentation Prof. D. Sarkar Department of Chemical Engineering Indian Institute of Technology, Kharagpur

Process Control and Instrumentation Prof. D. Sarkar Department of Chemical Engineering Indian Institute of Technology, Kharagpur Lecture - 38 Instrumentation: General Principle of Measurement Systems (Contd.)

Chapter 10. Key Ideas Correlation, Correlation Coefficient (r),

Chapter 0 Key Ideas Correlation, Correlation Coefficient (r), Section 0-: Overview We have already explored the basics of describing single variable data sets. However, when two quantitative variables

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written

Classical Physics Prof. V. Balakrishnan Department of Physics Indian Institution of Technology, Madras. Lecture No. # 13

Classical Physics Prof. V. Balakrishnan Department of Physics Indian Institution of Technology, Madras Lecture No. # 13 Now, let me formalize the idea of symmetry, what I mean by symmetry, what we mean

Wave Hydro Dynamics Prof. V. Sundar Department of Ocean Engineering Indian Institute of Technology, Madras

Wave Hydro Dynamics Prof. V. Sundar Department of Ocean Engineering Indian Institute of Technology, Madras Module No. # 02 Wave Motion and Linear Wave Theory Lecture No. # 03 Wave Motion II We will today

Lecture 3. Mathematical Induction

Lecture 3 Mathematical Induction Induction is a fundamental reasoning process in which general conclusion is based on particular cases It contrasts with deduction, the reasoning process in which conclusion

OLS is not only unbiased it is also the most precise (efficient) unbiased estimation technique - ie the estimator has the smallest variance

Lecture 5: Hypothesis Testing What we know now: OLS is not only unbiased it is also the most precise (efficient) unbiased estimation technique - ie the estimator has the smallest variance (if the Gauss-Markov

t-statistics for Weighted Means with Application to Risk Factor Models

t-statistics for Weighted Means with Application to Risk Factor Models Lisa R. Goldberg Barra, Inc. 00 Milvia St. Berkeley, CA 94704 Alec N. Kercheval Dept. of Mathematics Florida State University Tallahassee,

Basic Electronics Prof. Dr. Chitralekha Mahanta Department of Electronics and Communication Engineering Indian Institute of Technology, Guwahati

Basic Electronics Prof. Dr. Chitralekha Mahanta Department of Electronics and Communication Engineering Indian Institute of Technology, Guwahati Module: 2 Bipolar Junction Transistors Lecture-10 Transistor

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

Data and Regression Analysis. Lecturer: Prof. Duane S. Boning. Rev 10

Data and Regression Analysis Lecturer: Prof. Duane S. Boning Rev 10 1 Agenda 1. Comparison of Treatments (One Variable) Analysis of Variance (ANOVA) 2. Multivariate Analysis of Variance Model forms 3.

Fundamentals of Operations Research. Prof. G. Srinivasan. Department of Management Studies. Indian Institute of Technology, Madras. Lecture No.

Fundamentals of Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Lecture No. # 22 Inventory Models - Discount Models, Constrained Inventory

Correlation key concepts:

CORRELATION Correlation key concepts: Types of correlation Methods of studying correlation a) Scatter diagram b) Karl pearson s coefficient of correlation c) Spearman s Rank correlation coefficient d)

Lecture - 5 Zener Diode and Applications

Basic Electronics (Module 1 Semiconductor Diodes) Dr. Chitralekha Mahanta Department of Electronics and Communication Engineering Indian Institute of Technology, Guwahati Lecture - 5 Zener Diode and Applications

Determining When an Expression Is Undefined

Determining When an Expression Is Undefined Connections Have you ever... Tried to use a calculator to divide by zero and gotten an error Tried to figure out the square root of a negative number Expressions

Regression. Name: Class: Date: Multiple Choice Identify the choice that best completes the statement or answers the question.

Class: Date: Regression Multiple Choice Identify the choice that best completes the statement or answers the question. 1. Given the least squares regression line y8 = 5 2x: a. the relationship between

Lecture 16: Generalized Additive Models Regression III: Advanced Methods Bill Jacoby Michigan State University http://polisci.msu.edu/jacoby/icpsr/regress3 Goals of the Lecture Introduce Additive Models

Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay

Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 17 Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding