Linear Regression and Correlation

Transcription

1 Linear Regression and Correlation Business Statistics Plan for Today Bivariate analysis and linear regression Lines and linear equations Scatter plots Least-squares regression line Correlation coefficient Correlation does not imply causation Coefficient of determination Examples 1

2 Bivariate Analysis Is there a relationship between two variables? For example, is there a relationship between a person s income and his level of educational attainment? These types of questions are studied by the bivariate analysis, where bi indicates two variables. Statisticians also work with more than two variables, leading to a multivariate analysis. Linear Regression The simplest form of regression, "linear regression" has one independent variable (x). It looks for an equation of the line that best describes the relationship between two variables. This involves data that fits a line in two dimensions. Note: You will also study correlation (Correlation Coefficient) which measures how strong the relationship is. 2

3 Linear Equations y = a + bx where x is the independent variable, plotted on the horizontal axis, and y is the dependent variable, plotted on the vertical axis. The graph of this equation is a straight line. The numbers a and b are constants. The number a is the so-called y-intercept, and it shows where the line meets the vertical axis. The number b is the slope (Fr.: la pente), and it indicates the steepness of the line. Three types of slopes (a) b > 0 : the line goes up from left to right. (b) b = 0 : the line is horizontal. (c) b < 0 : the line goes down from left to right. 3

4 Understanding slope Example: the daily profit of a street vendor selling hot dogs is y = x, where x is the number of hot dogs sold and the profit y is measured in dollars. This equation implies that for each hot dog sold, the daily profit increases by $1.50. In general, the slope shows how the dependent variable y increases (or decreases) when the independent variable x increases by one unit. Scatter Plots A scatter plot is one of the most common ways to display a relationship between two variables, x and y. Things to look for in a scatter plot: Do the points seem to form a particular shape? Are the points close to a particular line, of positive or negative slope? Can you say there is a relationship between the values of x and y? 4

5 Scatter plots Scatter plots In this section, we will be interested in scatter plots that show some kinds of linear patterns. 5

6 Scatter plots If we think that the points show a linear relationship, we would like to draw a line on the scatter plot. This line can be calculated through a process called linear regression. However, we only calculate a regression line if one of the variables helps to explain or predict the other variable. If x is the independent variable and y the dependent variable, then we can use a regression line to predict y for a given value of x. Linear regression We look for a line that best fits the points on a scatter plot. The least-squares regression line or the trendline is the one that has the smallest sum of the squared errors. 6

7 Calculating the least-squares line Given n pairs of data x 1, y 1, x 2, y 2,, (x n, y n ) The equation of the least-squares line is Where the slope the y-intercept y = a + bx b = xy x y x 2 x 2 a = y b x Example of computation of the least-squares line Given the following six pairs of data (x, y): (-1,-2), (1,-1), (2,3), (4,3), (6,5), (7,8) We compute the values of xy: 2, -1, 6, 12, 30, 56 and the values of x 2 : 1, 1, 4, 16, 36, 49. Then we have: x = , y = x 2 = , and xy = 17.5 Thus, and b = xy x y = x 2 x 2 a = y b x =

8 Example (continued) Now we plot the data and the least-squares line on a graph: We can also compute a and b using a built-in function on a scientific calculator (selected models). Interpolation and extrapolation Knowing the equation of the trendline, we can predict the unknown y-values inside the range (interpolation) as well as outside the range (extrapolation). In the previous example: For the x-value of x = 3 we predict y For x = 5 we predict y Both are interpolations. For x = 3 we predict y For x = 10 we predict y These two are extrapolations. 8

9 Interpolations and extrapolations We can also interpolate (and extrapolate) backwards. Given a y-value, we can solve the linear regression equation and find the value of x. In the previous example, if y = 2, then we can predict x = (y a)/ b If y = 4, we can predict that x One must be careful, because extrapolations are rarely justified without additional information, and interpolations are justified only when correlation is strong (see below). Correlation Coefficient r It was developed by Karl Pearson in the early 1900s as a numerical measure of strength and direction of the linear association between the independent variable x and the dependent variable y. The value of r is always between 1 and 1. If r > 0, the correlation is positive (when x increases, y increases as well). If r < 0, the correlation is negative (when x increases, y decreases). 9

10 Correlation Coefficient r r 1 : perfect negative correlation 1 < r < 0.6 : strong negative correlation 0.6 < r < 0.3 : moderate negative correlation 0.3 < r < 0 : weak negative correlation r 0 : no correlation 0 < r < 0.3 : weak positive correlation 0.3 < r < 0.6 : moderate positive correlation 0.6 < r < 1 : strong positive correlation r 1 : perfect positive correlation Correlation does not imply causation Even if there is a strong positive correlation between x and y, we cannot say that x implies y, nor that y implies x. Example: there is a strong positive correlation between the level of sales of ice cream and the number of drowning deaths. Does ice cream cause drowning deaths? Do drowning deaths cause increased consumption of ice cream? Most likely not, there is a 3 rd common factor: summer and hot weather. 10

11 The Coefficient of Determination r 2 It is equal to the square of the correlation coefficient, and can be stated as a percent, as well as in decimal form. The coefficient of determination r 2 has an interpretation in the context of the data: it represents the percentage of variation in the dependent (predicted) variable y that can be explained by variation in the independent variable x using the regression (best-fit) line. Example Suppose that the correlation coefficient between a person s salary and his or her educational attainment is equal to The coefficient of determination is therefore r 2 = We can say that approximately 41% of the variation in the person s salary can be explained by the variation in his or her educational attainment, using the best-fit regression line. 11

12 A formula for the coefficient of correlation r = s x s y b where s x and s y are the standard deviations for the x- and y- data respectively. Recall that and s x = s y = x i n 1 x 2 y i y 2 n 1 Earlier example (continued) Given the following six pairs of data (x, y): (-1,-2), (1,-1), (2,3), (4,3), (6,5), (7,8) We compute s x = and s y = Thus, r = s x s y b = and r 2 = This is an example of a strong positive correlation. This also can be computed on a scientific calculator. 12

13 In-class exercise: salaries and education The table on the left represents a sample of 11 individuals working for the government of Quebec, their annual salary in thousands of $, and their educational attainment, in years. Scatter Plot 13

14 Computing the trendline etc. Using the formulas from before or the built-in calculator functions, compute: a = and b = r = This is an example of a strong positive correlation. r 2 = (what does it mean?) The slope of the trendline (or the regression line) equals what does it mean? Adding trendline (least squares line) 14

15 Adding the equation and the coefficient of determination Using trendline for predictions How can the trendline be used to predict the most probable values of y for other values of x? Use the equation: y = x Each value of x yields a prediction for the corresponding value of y. For example, someone with 18 years of scholarity is predicted to make 84.7 thousand $ per year working for the government. Of course, the real figure can be higher or lower: our prediction is based on a small sample. 15

16 Final Remarks Watch Youtube video: Homework Upcoming Test 16