# Simple Regression Theory II 2010 Samuel L. Baker

Save this PDF as:

Size: px
Start display at page:

## Transcription

2 SIMPLE REGRESSION THEORY II 2 In assignment 1, everybody in the class had data with the same true slope and intercept, but different errors. As a result of the different errors, everybody got different values for ^ and ^. That is what it means to say that the estimates of and are random variables. Because each estimate is a random variable, each estimator (recipe for getting the estimate) has what is called a sampling distribution. This means that each estimator, when used in a particular situation, has an expected value and a variance expected spread. A good estimator a good way of making an estimate would be one that is likely to give an estimate that is close to the true value. A good estimator would have an expected value that is near the true value and a variance that is small. As mentioned, Assumption 1 is that there is a true line, and that the observed data points scatter around that line due to random error. By adding more assumptions, we can use the sampling distribution idea to move toward assessing how good an estimator the least squares method is. We can also use this idea to assess how good an estimator the eyeball method is. Assumption 2: The expected value of any error is 0. We can write this assumption in algebraic terms as: Expected(e i ) = 0 for all each observation i. (i numbers the observations from 1 to N), This means that we assume that the points we observe are not systematically above or below the true line. If, in practice, there are more points, say, above the true line than below it, that is entirely due to random error. If this assumption is true, then the expected values of the least squares line's slope and intercept are the true line's slope and intercept. Algebraically, we can write: Expected( ^ ) = and Expected( ^) =. Those formulas mean that the least squares regression line is just as likely to be above the true line as below it, and just as likely to be too steep as to be not steep enough. The slope and intercept estimates are aimed at their targets. The jargon term for this is that the estimators are unbiased. The next two assumptions are about the variances and covariances of the observations errors (e i ). Assumption 3: All the errors have the same variance. Expected(e i2 ) = 2 for all i Expected(e i2 ) is the variance of ith data point s error e i. The variance formula usually involves subtracting the mean, but here the mean of each error is 0. That is assumption 2, that the expected value of e i is 0. Notice that, in the formula Expected(e i2 ) = 2, there is a subscript i on the left side of the equals sign, but no i subscript on the right side. This expresses the idea that all the errors variances are the same. I use 2 ("sigma-squared") here for the variance of the error, because that is the textbook convention.

3 SIMPLE REGRESSION THEORY II 3 All the errors having the same variance means that all of the observations are equally likely to be far from or near to the true line. No one observation is more reliable than any other, deserving more weight. In almost any data set, some data points will be closer to the true line than others. This, we assume, is entirely by luck. If one or two points stick way out, like a basketball player in a room full of jockeys, we have doubts about assumption 3. Assumption 4: All the errors are independent of each other. Expected(e i e j ) = 0 for any two observations such that i =/ j Expected(e i e j ) is the covariance of the two random variables e i and e j. The assumption that this is 0 for any pair of observations means this: If the one point happens to be above the true line, it is not any more or less likely that the next point will also be above the true line. A comment on making these assumptions These assumptions 1 through 4 are not made just from the data. Rather, these assumptions must come primarily from our general understanding of how the data were generated. We can look at the data and get an idea about how reasonable these assumptions are. (Assignment 2 does this.) Usually, we pick the assumptions we make based on a combination of the look of the data and our understanding of where the data came from. When you fit a least squares line to some points, you are implicitly making assumptions 1 through 4. When you draw a straight line by eye through a bunch of points, you are also implicitly making assumptions 1 through 4. What if you are not comfortable with making those assumptions? Later in the course, we will get into some of the alternative models that you can try when one of more of these assumptions do not hold. For example, we will talk about non-linear models. For now, let us stick with the linear model, which means that we accept those assumptions. Variances of the least squares intercept and slope estimates Earlier, it was pointed out that the estimates for the slope and intercept, ^ and ^, are random variables, so each has a mean, or expected value, and a variance. From assumptions 1 through 4, one can derive formulas for the variances of ^ and ^ when they are estimated using least squares. (You cannot derive formulas for the variances of ^ and ^ when they are estimated using the draw-by-eye method. To estimate those variances, you could ask a number of people to draw regression lines by eye. That is what this class does for Assignment 1!) Here are formulas for the variances of the least squares estimators of ^ and ^. (If you would like to see how these formulas are derived, please consult a statistics textbook.)

4 SIMPLE REGRESSION THEORY II 4 In these formulas, 2 is the variance of the errors. Asumption 3 says all errors have the same variance, which is 2. One of the themes of this course is that you can learn something from formulas like these, if you take the trouble to examine them. Don t let them intimidate you! Each formula has 2 multiplied or divided by something. This means that the variances of our regression line parameters are both proportional to 2, the variance of the errors from the true equation. An implication is that the smaller the errors are, the less randomness there is in our ^ and ^ values, and the closer our estimated parameters are likely to be to the true parameters. In the formula for the variance of ^, the denominator gets bigger when there are more X's and when those X's are more spread out. A bigger denominator makes a fraction smaller. This tells us something about designing a study or experiment: If you want your estimates to come out close to the truth, get a lot of observations that are spread out over a big range of values of the independent variable. Those formulas have a major drawback: We cannot directly use them! That is because we don't know what 2 is. What we can do is estimate 2. We call our estimate s 2, and calculate it from the residuals like this: In that expression, the residuals are designated by u i. The numerator says to square each residual and then add them all up, giving you the sum of the squares of the residuals. (The least squares line is the line that makes this sum the smallest.) To get the estimate of the variance of the errors, we divide the sum of squared residuals by N 2. That makes the estimated variance like an average squared residual. I say like an average squared residual because we divide by N 2, rather than by N. Why divide by N-2, instead of N? Using N-2 allows for the fact that the least squares line will fit the data better than the true line does. The least squares line is, by the definition of least squares, the line with the smallest sum of squared residuals. Any other line, including the true line, will have a larger sum of squared residuals. N-2 is the degrees of freedom. In your introductory statistics course, you used models in which the degrees of freedom is N-1. For instance, when you estimated a population variance based on the data in a sample of N from that population, the degrees of freedom was N-1. Why did you subtract 1 from N then, and why are we subtracting 2 now? Then, to estimate the variance of a population around the population s mean, you used one statistic, the mean of the sample. Now, we are using predicted values calculated from two statistics, the slope and the intercept of the regression line. It s easier to get a predicted value that is close to your observation when you have two estimated parameters, instead of just one. Using N 2 allows for that. In general, the number of degrees of freedom is N-P, where P is the number of parameters used to get the number(s) that you subtract from each observation s value.

5 SIMPLE REGRESSION THEORY II 5 To repeat, the reason you subtract P from N in these situations is that the estimated mean and the estimated regression line fit the data better than the actual population mean or the true line would. The more parameters you have, the easier it is to fit your model to the data, even if the model really is not any good. You have to make the degrees of freedom smaller to make the estimated variance come out big enough to correct for this. Substituting s 2 for 2 in the formulas for the variances of ^ and ^ gives estimated variances for ^ and ^. You just change every " " into an "s". Standard Error The standard error of something is the square root of its estimated variance. s, the square root of s 2, is called the standard error of the regression. Here is the formula: ^ has a standard error, too. It s the square root of the estimated variance of ^:: This measures of how much risk there is in using ^ as an estimate of. You can deduce from the formula that the risk in the estimate of is smaller if: the actual points lie close to the regression line (so that s is small), or the X's are far apart (so that the sum of squared X deviations is big). Hypothesis tests on the slope parameter To do conventional hypothesis testing and confidence intervals, we must make another assumption about the errors, one that is even more restrictive: Assumption 5 (needed for hypothesis testing): Each error has the normal distribution. Each error

8 SIMPLE REGRESSION THEORY II 8 The drawback of a small significance level is that it increases the probability of a Type 2 error. With a small significance level, it is more likely that you will fail to detect a true relationship between X and Y. With a small significance level, you are demanding overwhelming evidence of a relationship being there. The ideal way to pick a significance level is to weigh the consequences of each type of error, and pick a significance level that best balances the costs. What researchers often do, however, is pick.05, because they know that this significance level is generally accepted. Watch out for confusing or contradictory terminology: 1. -level of significance. The significance level is sometimes called the -level. Try not to confuse this with the intercept of a linear regression equation. 2. High significance level. A low alpha number is a high level of significance. If you reject the hypothesis that a coefficient is 0 at the level, that coefficient is highly significant % significance or 0.05 significance. These terms are interchangeable. The same goes for 99% significance or 0.01 significance. Usually, the context will enable you keep these straight. Confidence intervals for equation parameters An alternative way to test hypotheses about is to calculate a confidence interval for ^ and then see if your hypothesized value is inside it. The 95% confidence interval (two-tailed test) for ^ is: 95% confidence interval for the slope estimate The t 0.05 in the formulas above means the value from the t table in the column for the 0.05 significance level for a two-tailed test and the row for N-2 degrees of freedom. For a 99% confidence interval, you would use t The ± means that you evaluate the expression once with a + to get the top of the confidence interval, then you evaluate it with a - to get the bottom. This confidence interval is called two-tailed because it has a high end and a low end that are equidistant from the estimated value. That is what the ± does. (A one-tailed confidence interval would have one end that is above the estimated value and go off to minus infinity to the left, or it could have one end that is below the estimated value and go off to plus infinity to the right.) Look now at the right hand version of the confidence interval formula above. Let s explore the underlying relationships. The s in the numerator tells us that the confidence intervals for the coefficients get bigger if the residuals are bigger. The denominator tells us that having more X values and having them more spread out from their mean makes the confidence interval smaller. Again, this tells us that we

9 SIMPLE REGRESSION THEORY II 9 gain confidence in our estimate if the residuals are small and if we have a lot of spread-out X values. For the intercept,, the 95% confidence interval is: Confidence interval for the prediction of Y We can also calculate confidence intervals for a prediction. Let X 0 be the X value for which we want the predicted Y value. We ll call that predicted Y value Y^ 0.. The 95% confidence interval for the prediction is: Here's what we can see in this expression: The confidence interval is the big expression to the right of the ±. The width of this confidence interval depends on the sum of squared residuals s and depends inversely on (X i -X.) 2, the sum of the squared deviations of the X values from their mean. Big residuals, relative to the spread of the X's, make s big, which makes the confidence intervals wide. More X values, or more spread-out X values, make (X i -X.) 2 expression larger, which makes the confidence interval narrower. The (X 0 -X ) 2 expression in the numerator under the square root sign tells us that confidence interval gets wider as the X 0 you choose gets further from the mean of the X's. R 2, a measure of fit An overall measure of how well the regression line fits the points is R 2 (read "R squared"). R 2 is always between 0 (no fit) and 1 (perfect fit). R 2 depends on how big the residuals are in relation to the deviations of the Y's from their mean. It is customary to say that the R 2 tells you how much of the variation in the Y's is "explained" by the regression line.

10 SIMPLE REGRESSION THEORY II 10 Correlation Correlation is a measure of how much two variables are linearly related to each other. If two variables, X and Y, go up and down together in more or less a straight line way, they are positively correlated. If Y goes down when X goes up, they are negatively correlated. In a simple regression, the correlation is the square root of the regression s R 2. For this reason, the correlation (also called correlated coefficient ) is designated r. An alternative formula for r, equivalent to the square root of the R 2 is: The correlation coefficient The R 2 ranges from 0 to +1. r can range from -1 to +1 With some algebra, you can show that r and are related. Multiply the top and bottom of the r formula by the square root of the sum of the squares of the x deviations. You get this: Look at the parts of this fraction that are not under square root symbols. These are the left halves of the numerator and denominator. Do you recognize them as the same as the formula for the least squares ^?

11 SIMPLE REGRESSION THEORY II 11 The higher the slope of the regression line is (higher absolute value of ^ ), the higher r is. When X and Y are not correlated, r = 0. According to the formula above, ^ must be 0 as well. So, if X and Y are unrelated, the least squares line is horizontal. Durbin-Watson Statistic In Assignment 2 you will see that the regular goodness of fit statistics (R 2 and t) cannot detect situations where a linear least squares model is not appropriate. The Durbin-Watson statistic can detect some of those situations. In particular, the Durbin-Watson statistic tests for serial (as in "series") correlation of the residuals. Here is the Durbin-Watson statistic formula: (The u s in this formula are the residuals.) A rule-of-thumb for interpretation of DW: DW < 1 indicates residuals track each other. A positive residual tends to be followed by another positive residual. A negative residual tends to be followed by another negative residual. DW near 2 indicates no serial correlation. DW > 3 indicates residuals alternate, positive-negative-positive-negative.. For a more formal test, see the Durbin-Watson table in the downloadable file of tables. Serial correlation of residuals indicates that you can do better than your current model at predicting. Least Squares assumes that the next residual will be 0. If there is serial correlation, that means that you can partly predict a residual from the one that came before. If the Durbin-Watson test finds serial correlation, it may indicate that a curved model would be better than a straight line model. Another possibility is that the true relationship is a line, but the error in one observation is affecting the error in the next observation. If this is so, you can get better predictions with a more elaborate model that takes this effect into account.

### The Philosophy of Hypothesis Testing, Questions and Answers 2006 Samuel L. Baker

HYPOTHESIS TESTING PHILOSOPHY 1 The Philosophy of Hypothesis Testing, Questions and Answers 2006 Samuel L. Baker Question: So I'm hypothesis testing. What's the hypothesis I'm testing? Answer: When you're

### Simple Regression Theory I 2010 Samuel L. Baker

SIMPLE REGRESSION THEORY I 1 Simple Regression Theory I 2010 Samuel L. Baker Regression analysis lets you use data to explain and predict. A simple regression line drawn through data points In Assignment

### Multiple Regression Theory 2006 Samuel L. Baker

MULTIPLE REGRESSION THEORY 1 Multiple Regression Theory 2006 Samuel L. Baker Multiple regression is regression with two or more independent variables on the right-hand side of the equation. Use multiple

### Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur. Lecture - 2 Simple Linear Regression

Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur Lecture - 2 Simple Linear Regression Hi, this is my second lecture in module one and on simple

### Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression

Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression Objectives: To perform a hypothesis test concerning the slope of a least squares line To recognize that testing for a

### NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( )

Chapter 340 Principal Components Regression Introduction is a technique for analyzing multiple regression data that suffer from multicollinearity. When multicollinearity occurs, least squares estimates

### 1) Write the following as an algebraic expression using x as the variable: Triple a number subtracted from the number

1) Write the following as an algebraic expression using x as the variable: Triple a number subtracted from the number A. 3(x - x) B. x 3 x C. 3x - x D. x - 3x 2) Write the following as an algebraic expression

### " Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

### Simple Linear Regression Chapter 11

Simple Linear Regression Chapter 11 Rationale Frequently decision-making situations require modeling of relationships among business variables. For instance, the amount of sale of a product may be related

### e = random error, assumed to be normally distributed with mean 0 and standard deviation σ

1 Linear Regression 1.1 Simple Linear Regression Model The linear regression model is applied if we want to model a numeric response variable and its dependency on at least one numeric factor variable.

### Introduction to Regression. Dr. Tom Pierce Radford University

Introduction to Regression Dr. Tom Pierce Radford University In the chapter on correlational techniques we focused on the Pearson R as a tool for learning about the relationship between two variables.

### Inferential Statistics

Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

### CALCULATIONS & STATISTICS

CALCULATIONS & STATISTICS CALCULATION OF SCORES Conversion of 1-5 scale to 0-100 scores When you look at your report, you will notice that the scores are reported on a 0-100 scale, even though respondents

### OLS is not only unbiased it is also the most precise (efficient) unbiased estimation technique - ie the estimator has the smallest variance

Lecture 5: Hypothesis Testing What we know now: OLS is not only unbiased it is also the most precise (efficient) unbiased estimation technique - ie the estimator has the smallest variance (if the Gauss-Markov

### Unit 21 Student s t Distribution in Hypotheses Testing

Unit 21 Student s t Distribution in Hypotheses Testing Objectives: To understand the difference between the standard normal distribution and the Student's t distributions To understand the difference between

### AMS7: WEEK 8. CLASS 1. Correlation Monday May 18th, 2015

AMS7: WEEK 8. CLASS 1 Correlation Monday May 18th, 2015 Type of Data and objectives of the analysis Paired sample data (Bivariate data) Determine whether there is an association between two variables This

### Chapter 7 Part 2. Hypothesis testing Power

Chapter 7 Part 2 Hypothesis testing Power November 6, 2008 All of the normal curves in this handout are sampling distributions Goal: To understand the process of hypothesis testing and the relationship

### 17. SIMPLE LINEAR REGRESSION II

17. SIMPLE LINEAR REGRESSION II The Model In linear regression analysis, we assume that the relationship between X and Y is linear. This does not mean, however, that Y can be perfectly predicted from X.

### Hypothesis testing - Steps

Hypothesis testing - Steps Steps to do a two-tailed test of the hypothesis that β 1 0: 1. Set up the hypotheses: H 0 : β 1 = 0 H a : β 1 0. 2. Compute the test statistic: t = b 1 0 Std. error of b 1 =

### Week 4: Standard Error and Confidence Intervals

Health Sciences M.Sc. Programme Applied Biostatistics Week 4: Standard Error and Confidence Intervals Sampling Most research data come from subjects we think of as samples drawn from a larger population.

### 1. What is the critical value for this 95% confidence interval? CV = z.025 = invnorm(0.025) = 1.96

1 Final Review 2 Review 2.1 CI 1-propZint Scenario 1 A TV manufacturer claims in its warranty brochure that in the past not more than 10 percent of its TV sets needed any repair during the first two years

### Regression Analysis: Basic Concepts

The simple linear model Regression Analysis: Basic Concepts Allin Cottrell Represents the dependent variable, y i, as a linear function of one independent variable, x i, subject to a random disturbance

### Chapter 10. Key Ideas Correlation, Correlation Coefficient (r),

Chapter 0 Key Ideas Correlation, Correlation Coefficient (r), Section 0-: Overview We have already explored the basics of describing single variable data sets. However, when two quantitative variables

### Box plots & t-tests. Example

Box plots & t-tests Box Plots Box plots are a graphical representation of your sample (easy to visualize descriptive statistics); they are also known as box-and-whisker diagrams. Any data that you can

Section 5.4 The Quadratic Formula 481 5.4 The Quadratic Formula Consider the general quadratic function f(x) = ax + bx + c. In the previous section, we learned that we can find the zeros of this function

### Definition 8.1 Two inequalities are equivalent if they have the same solution set. Add or Subtract the same value on both sides of the inequality.

8 Inequalities Concepts: Equivalent Inequalities Linear and Nonlinear Inequalities Absolute Value Inequalities (Sections 4.6 and 1.1) 8.1 Equivalent Inequalities Definition 8.1 Two inequalities are equivalent

### Lecture - 32 Regression Modelling Using SPSS

Applied Multivariate Statistical Modelling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur Lecture - 32 Regression Modelling Using SPSS (Refer

### A correlation exists between two variables when one of them is related to the other in some way.

Lecture #10 Chapter 10 Correlation and Regression The main focus of this chapter is to form inferences based on sample data that come in pairs. Given such paired sample data, we want to determine whether

### Section P.9 Notes Page 1 P.9 Linear Inequalities and Absolute Value Inequalities

Section P.9 Notes Page P.9 Linear Inequalities and Absolute Value Inequalities Sometimes the answer to certain math problems is not just a single answer. Sometimes a range of answers might be the answer.

### Homework #3 is due Friday by 5pm. Homework #4 will be posted to the class website later this week. It will be due Friday, March 7 th, at 5pm.

Homework #3 is due Friday by 5pm. Homework #4 will be posted to the class website later this week. It will be due Friday, March 7 th, at 5pm. Political Science 15 Lecture 12: Hypothesis Testing Sampling

### CHAPTER 13 SIMPLE LINEAR REGRESSION. Opening Example. Simple Regression. Linear Regression

Opening Example CHAPTER 13 SIMPLE LINEAR REGREION SIMPLE LINEAR REGREION! Simple Regression! Linear Regression Simple Regression Definition A regression model is a mathematical equation that descries the

### LAB 4 INSTRUCTIONS CONFIDENCE INTERVALS AND HYPOTHESIS TESTING

LAB 4 INSTRUCTIONS CONFIDENCE INTERVALS AND HYPOTHESIS TESTING In this lab you will explore the concept of a confidence interval and hypothesis testing through a simulation problem in engineering setting.

### Using Excel for inferential statistics

FACT SHEET Using Excel for inferential statistics Introduction When you collect data, you expect a certain amount of variation, just caused by chance. A wide variety of statistical tests can be applied

### Sampling and Hypothesis Testing

Population and sample Sampling and Hypothesis Testing Allin Cottrell Population : an entire set of objects or units of observation of one sort or another. Sample : subset of a population. Parameter versus

### 2. Simple Linear Regression

Research methods - II 3 2. Simple Linear Regression Simple linear regression is a technique in parametric statistics that is commonly used for analyzing mean response of a variable Y which changes according

### Biostatistics: DESCRIPTIVE STATISTICS: 2, VARIABILITY

Biostatistics: DESCRIPTIVE STATISTICS: 2, VARIABILITY 1. Introduction Besides arriving at an appropriate expression of an average or consensus value for observations of a population, it is important to

### Simple Regression and Correlation

Simple Regression and Correlation Today, we are going to discuss a powerful statistical technique for examining whether or not two variables are related. Specifically, we are going to talk about the ideas

### E205 Final: Version B

Name: Class: Date: E205 Final: Version B Multiple Choice Identify the choice that best completes the statement or answers the question. 1. The owner of a local nightclub has recently surveyed a random

### The correlation coefficient

The correlation coefficient Clinical Biostatistics The correlation coefficient Martin Bland Correlation coefficients are used to measure the of the relationship or association between two quantitative

### Pearson's Correlation Tests

Chapter 800 Pearson's Correlation Tests Introduction The correlation coefficient, ρ (rho), is a popular statistic for describing the strength of the relationship between two variables. The correlation

### Numerical Summarization of Data OPRE 6301

Numerical Summarization of Data OPRE 6301 Motivation... In the previous session, we used graphical techniques to describe data. For example: While this histogram provides useful insight, other interesting

### Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

### Estimation of σ 2, the variance of ɛ

Estimation of σ 2, the variance of ɛ The variance of the errors σ 2 indicates how much observations deviate from the fitted surface. If σ 2 is small, parameters β 0, β 1,..., β k will be reliably estimated

### AP Statistics 2011 Scoring Guidelines

AP Statistics 2011 Scoring Guidelines The College Board The College Board is a not-for-profit membership association whose mission is to connect students to college success and opportunity. Founded in

### 17.0 Linear Regression

17.0 Linear Regression 1 Answer Questions Lines Correlation Regression 17.1 Lines The algebraic equation for a line is Y = β 0 + β 1 X 2 The use of coordinate axes to show functional relationships was

### Confidence intervals, t tests, P values

Confidence intervals, t tests, P values Joe Felsenstein Department of Genome Sciences and Department of Biology Confidence intervals, t tests, P values p.1/31 Normality Everybody believes in the normal

### Outline. Correlation & Regression, III. Review. Relationship between r and regression

Outline Correlation & Regression, III 9.07 4/6/004 Relationship between correlation and regression, along with notes on the correlation coefficient Effect size, and the meaning of r Other kinds of correlation

### MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS

MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS MSR = Mean Regression Sum of Squares MSE = Mean Squared Error RSS = Regression Sum of Squares SSE = Sum of Squared Errors/Residuals α = Level of Significance

### Statistical Functions in Excel

Statistical Functions in Excel There are many statistical functions in Excel. Moreover, there are other functions that are not specified as statistical functions that are helpful in some statistical analyses.

### Independent samples t-test. Dr. Tom Pierce Radford University

Independent samples t-test Dr. Tom Pierce Radford University The logic behind drawing causal conclusions from experiments The sampling distribution of the difference between means The standard error of

### Simple linear regression

Simple linear regression Introduction Simple linear regression is a statistical method for obtaining a formula to predict values of one variable from another where there is a causal relationship between

### Partial Fractions. Combining fractions over a common denominator is a familiar operation from algebra:

Partial Fractions Combining fractions over a common denominator is a familiar operation from algebra: From the standpoint of integration, the left side of Equation 1 would be much easier to work with than

### AP Statistics 1998 Scoring Guidelines

AP Statistics 1998 Scoring Guidelines These materials are intended for non-commercial use by AP teachers for course and exam preparation; permission for any other use must be sought from the Advanced Placement

### Comparing Means in Two Populations

Comparing Means in Two Populations Overview The previous section discussed hypothesis testing when sampling from a single population (either a single mean or two means from the same population). Now we

### The aspect of the data that we want to describe/measure is the degree of linear relationship between and The statistic r describes/measures the degree

PS 511: Advanced Statistics for Psychological and Behavioral Research 1 Both examine linear (straight line) relationships Correlation works with a pair of scores One score on each of two variables ( and

### Case Study in Data Analysis Does a drug prevent cardiomegaly in heart failure?

Case Study in Data Analysis Does a drug prevent cardiomegaly in heart failure? Harvey Motulsky hmotulsky@graphpad.com This is the first case in what I expect will be a series of case studies. While I mention

### Statistics courses often teach the two-sample t-test, linear regression, and analysis of variance

2 Making Connections: The Two-Sample t-test, Regression, and ANOVA In theory, there s no difference between theory and practice. In practice, there is. Yogi Berra 1 Statistics courses often teach the two-sample

### AP Physics 1 and 2 Lab Investigations

AP Physics 1 and 2 Lab Investigations Student Guide to Data Analysis New York, NY. College Board, Advanced Placement, Advanced Placement Program, AP, AP Central, and the acorn logo are registered trademarks

### SIMPLE REGRESSION ANALYSIS

SIMPLE REGRESSION ANALYSIS Introduction. Regression analysis is used when two or more variables are thought to be systematically connected by a linear relationship. In simple regression, we have only two

### The Dummy s Guide to Data Analysis Using SPSS

The Dummy s Guide to Data Analysis Using SPSS Mathematics 57 Scripps College Amy Gamble April, 2001 Amy Gamble 4/30/01 All Rights Rerserved TABLE OF CONTENTS PAGE Helpful Hints for All Tests...1 Tests

### , has mean A) 0.3. B) the smaller of 0.8 and 0.5. C) 0.15. D) which cannot be determined without knowing the sample results.

BA 275 Review Problems - Week 9 (11/20/06-11/24/06) CD Lessons: 69, 70, 16-20 Textbook: pp. 520-528, 111-124, 133-141 An SRS of size 100 is taken from a population having proportion 0.8 of successes. An

### Association Between Variables

Contents 11 Association Between Variables 767 11.1 Introduction............................ 767 11.1.1 Measure of Association................. 768 11.1.2 Chapter Summary.................... 769 11.2 Chi

### Example: Boats and Manatees

Figure 9-6 Example: Boats and Manatees Slide 1 Given the sample data in Table 9-1, find the value of the linear correlation coefficient r, then refer to Table A-6 to determine whether there is a significant

### SELF-TEST: SIMPLE REGRESSION

ECO 22000 McRAE SELF-TEST: SIMPLE REGRESSION Note: Those questions indicated with an (N) are unlikely to appear in this form on an in-class examination, but you should be able to describe the procedures

### Part 2: Analysis of Relationship Between Two Variables

Part 2: Analysis of Relationship Between Two Variables Linear Regression Linear correlation Significance Tests Multiple regression Linear Regression Y = a X + b Dependent Variable Independent Variable

### Chapter 11: Two Variable Regression Analysis

Department of Mathematics Izmir University of Economics Week 14-15 2014-2015 In this chapter, we will focus on linear models and extend our analysis to relationships between variables, the definitions

### Exercise 1.12 (Pg. 22-23)

Individuals: The objects that are described by a set of data. They may be people, animals, things, etc. (Also referred to as Cases or Records) Variables: The characteristics recorded about each individual.

### ALGEBRA I A PLUS COURSE OUTLINE

ALGEBRA I A PLUS COURSE OUTLINE OVERVIEW: 1. Operations with Real Numbers 2. Equation Solving 3. Word Problems 4. Inequalities 5. Graphs of Functions 6. Linear Functions 7. Scatterplots and Lines of Best

### Analysis of Variance ANOVA

Analysis of Variance ANOVA Overview We ve used the t -test to compare the means from two independent groups. Now we ve come to the final topic of the course: how to compare means from more than two populations.

### Technology Step-by-Step Using StatCrunch

Technology Step-by-Step Using StatCrunch Section 1.3 Simple Random Sampling 1. Select Data, highlight Simulate Data, then highlight Discrete Uniform. 2. Fill in the following window with the appropriate

### An analysis appropriate for a quantitative outcome and a single quantitative explanatory. 9.1 The model behind linear regression

Chapter 9 Simple Linear Regression An analysis appropriate for a quantitative outcome and a single quantitative explanatory variable. 9.1 The model behind linear regression When we are examining the relationship

### Hence, multiplying by 12, the 95% interval for the hourly rate is (965, 1435)

Confidence Intervals for Poisson data For an observation from a Poisson distribution, we have σ 2 = λ. If we observe r events, then our estimate ˆλ = r : N(λ, λ) If r is bigger than 20, we can use this

### Simple Linear Regression in SPSS STAT 314

Simple Linear Regression in SPSS STAT 314 1. Ten Corvettes between 1 and 6 years old were randomly selected from last year s sales records in Virginia Beach, Virginia. The following data were obtained,

### Regression. Name: Class: Date: Multiple Choice Identify the choice that best completes the statement or answers the question.

Class: Date: Regression Multiple Choice Identify the choice that best completes the statement or answers the question. 1. Given the least squares regression line y8 = 5 2x: a. the relationship between

### How To Run Statistical Tests in Excel

How To Run Statistical Tests in Excel Microsoft Excel is your best tool for storing and manipulating data, calculating basic descriptive statistics such as means and standard deviations, and conducting

BIOSTATISTICS QUIZ ANSWERS 1. When you read scientific literature, do you know whether the statistical tests that were used were appropriate and why they were used? a. Always b. Mostly c. Rarely d. Never

### Hints for Success on the AP Statistics Exam. (Compiled by Zack Bigner)

Hints for Success on the AP Statistics Exam. (Compiled by Zack Bigner) The Exam The AP Stat exam has 2 sections that take 90 minutes each. The first section is 40 multiple choice questions, and the second

### Study Resources For Algebra I. Unit 1C Analyzing Data Sets for Two Quantitative Variables

Study Resources For Algebra I Unit 1C Analyzing Data Sets for Two Quantitative Variables This unit explores linear functions as they apply to data analysis of scatter plots. Information compiled and written

### 3.4 Limits at Infinity - Asymptotes

3.4 Limits at Infinity - Asymptotes Definition 3.3. If f is a function defined on some interval (a, ), then f(x) = L means that values of f(x) are very close to L (keep getting closer to L) as x. The line

### Introduction to Finite Systems: Z 6 and Z 7

Introduction to : Z 6 and Z 7 The main objective of this discussion is to learn more about solving linear and quadratic equations. The reader is no doubt familiar with techniques for solving these equations

### Session 7 Bivariate Data and Analysis

Session 7 Bivariate Data and Analysis Key Terms for This Session Previously Introduced mean standard deviation New in This Session association bivariate analysis contingency table co-variation least squares

### Introduction to. Hypothesis Testing CHAPTER LEARNING OBJECTIVES. 1 Identify the four steps of hypothesis testing.

Introduction to Hypothesis Testing CHAPTER 8 LEARNING OBJECTIVES After reading this chapter, you should be able to: 1 Identify the four steps of hypothesis testing. 2 Define null hypothesis, alternative

### Wooldridge, Introductory Econometrics, 4th ed. Multiple regression analysis:

Wooldridge, Introductory Econometrics, 4th ed. Chapter 4: Inference Multiple regression analysis: We have discussed the conditions under which OLS estimators are unbiased, and derived the variances of

### The Basics of a Hypothesis Test

Overview The Basics of a Test Dr Tom Ilvento Department of Food and Resource Economics Alternative way to make inferences from a sample to the Population is via a Test A hypothesis test is based upon A

### The method of Least Squares using the Excel Solver

The method of Least Squares using the Excel Solver Michael Wood (Michael.wood@port.ac.uk) 22 October 2012 (Introduction and links to electronic versions of this document and the other parts at http://woodm.myweb.port.ac.uk/stats.

### Assessing Forecasting Error: The Prediction Interval. David Gerbing School of Business Administration Portland State University

Assessing Forecasting Error: The Prediction Interval David Gerbing School of Business Administration Portland State University November 27, 2015 Contents 1 Prediction Intervals 1 1.1 Modeling Error..................................

### Unit 29 Chi-Square Goodness-of-Fit Test

Unit 29 Chi-Square Goodness-of-Fit Test Objectives: To perform the chi-square hypothesis test concerning proportions corresponding to more than two categories of a qualitative variable To perform the Bonferroni

### Statistics 112 Regression Cheatsheet Section 1B - Ryan Rosario

Statistics 112 Regression Cheatsheet Section 1B - Ryan Rosario I have found that the best way to practice regression is by brute force That is, given nothing but a dataset and your mind, compute everything

### TIME SERIES REGRESSION

DEPARTMENT OF POLITICAL SCIENCE AND INTERNATIONAL RELATIONS Posc/Uapp 816 TIME SERIES REGRESSION I. AGENDA: A. A couple of general considerations in analyzing time series data B. Intervention analysis

### fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson

fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson Contents What Are These Demos About? How to Use These Demos If This Is Your First Time Using Fathom Tutorial: An Extended Example

### Measuring the Power of a Test

Textbook Reference: Chapter 9.5 Measuring the Power of a Test An economic problem motivates the statement of a null and alternative hypothesis. For a numeric data set, a decision rule can lead to the rejection

### Uncertainties and Error Propagation

Uncertainties and Error Propagation By Vern Lindberg 1. Systematic and random errors. No measurement made is ever exact. The accuracy (correctness) and precision (number of significant figures) of a measurement

### ModuMath Algebra Lessons

ModuMath Algebra Lessons Program Title 1 Getting Acquainted With Algebra 2 Order of Operations 3 Adding & Subtracting Algebraic Expressions 4 Multiplying Polynomials 5 Laws of Algebra 6 Solving Equations

### Statistical Inference

Statistical Inference Idea: Estimate parameters of the population distribution using data. How: Use the sampling distribution of sample statistics and methods based on what would happen if we used this

### PASS Sample Size Software. Linear Regression

Chapter 855 Introduction Linear regression is a commonly used procedure in statistical analysis. One of the main objectives in linear regression analysis is to test hypotheses about the slope (sometimes

### Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)

Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

### Chapter 7: Simple linear regression Learning Objectives

Chapter 7: Simple linear regression Learning Objectives Reading: Section 7.1 of OpenIntro Statistics Video: Correlation vs. causation, YouTube (2:19) Video: Intro to Linear Regression, YouTube (5:18) -

### PPS TI-83 Activities Algebra 1-2 Teacher Notes

PPS TI-83 Activities Algebra 1-2 Teacher Notes It is an expectation in Portland Public Schools that all teachers are using the TI-83 graphing calculator in meaningful ways in Algebra 1-2. This does not