Lecture 5 Hypothesis Testing in Multiple Linear Regression


 Dustin Adams
 2 years ago
 Views:
Transcription
1 Lecture 5 Hypothesis Testing in Multiple Linear Regression BIOST 515 January 20, 2004
2 Types of tests 1 Overall test Test for addition of a single variable Test for addition of a group of variables
3 Overall test 2 y i = β 0 + x i1 β x ip β p + ɛ i Does the entire set of independent variables contribute significantly to the prediction of y?
4 Test for an addition of a single variable 3 Does the addition of one particular variable of interest add significantly to the prediction of y acheived by the other independent variables already in the model? y i = β 0 + x i1 β x ip β p + ɛ i
5 Test for addition of a group of variables 4 Does the addition of some group of independent variables of interest add significantly to the prediction of y obtained through other independent variables already in the model? y i = β 0 + x i1 β x i,p 1 β p 1 + x ip β p + ɛ i
6 The ANOVA table 5 Source of Sums of squares Degrees of Mean E[Mean square] variation freedom square Regression SSR = ˆβ X y nȳ 2 SSR p p pσ 2 + β R X C X Cβ R Error SSE = y y ˆβ X SSE y n (p + 1) n (p+1) σ 2 Total SST O = y y nȳ 2 n 1 X C is the matrix of centered predictors: X C = 0 x 11 x 1 x 12 x 2 x 1p x p x 21 x 1. x 22 x 2. x 2p x p. x n1 x 1 x n2 x 2 x np x p 1 C A and β R = (β 1,, β p ).
7 The ANOVA table for 6 y i = β 0 + x i1 β1 + x i2 β2 + + x ip β p + ɛ i is often provided in the output from statistical software as Source of Sums of squares Degrees of F variation freedom Regression x 1 1 x 2 x 1. 1 x p x p 1, x p 2,, x 1 1 Error SSE n (p + 1) Total SST O n 1 where SSR = SSR(x 1 ) + SSR(x 2 x 1 ) + + SSR(x p x p 1, x p 2,..., x 1 ) and has p degrees of freedom.
8 Overall test 7 H 0 : β 1 = β 2 = = β p = 0 H 1 : β j 0 for at least one j, j = 1,..., p Rejection of H 0 implies that at least one of the regressors, x 1, x 2,..., x p, contributes significantly to the model. We will use a generalization of the Ftest in simple linear regression to test this hypothesis.
9 Under the null hypothesis, SSR/σ 2 χ 2 p and SSE/σ 2 χ 2 n (p+1) are independent. Therefore, we have 8 F 0 = SSR/p SSE/(n p 1) = MSR MSE F p,n p 1 Note: as in simple linear regression, we are assuming that ɛ i N(0, σ 2 ) or relying on large sample theory.
10 CHS example, cont. 9 > anova(lmwtht) Analysis of Variance Table y i = β 0 + weight i β 1 + height i β 2 + ɛ i Response: DIABP Df Sum Sq Mean Sq F value Pr(>F) WEIGHT ** HEIGHT Residuals Signif. codes: 0 *** ** 0.01 * ( )/2 F 0 = = 5.59 > F 2,495,.95 = /495 We reject the null hypothesis at α =.05 and conclude that at least one of β 1 or β 2 is not equal to 0.
11 The overall F statistic is also available from the output of summary(). 10 > summary(lmwtht) Call: lm(formula = DIABP ~ WEIGHT + HEIGHT, data = chs) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e10 *** WEIGHT * HEIGHT Signif. codes: 0 *** ** 0.01 * Residual standard error: on 495 degrees of freedom Multiple RSquared: , Adjusted Rsquared: Fstatistic: on 2 and 495 DF, pvalue:
12 Tests on individual regression coefficients 11 Once we have determined that at least one of the regressors is important, a natural next question might be which one(s)? Important considerations: Is the increase in the regression sums of squares sufficient to warrant an additional predictor in the model? Additional predictors will increase the variance of ŷ  include only predictors that explain the response (note: we may not know this through hypothesis testing as confounders may not test significant but would still be necessary in the regression model). Adding an unimportant predictor may increase the residual mean square thereby reducing the usefulness of the model.
13 12 y i = β 0 + x i1 β x ij β j + + x ip β p + ɛ i H 0 : β j = 0 H 1 : β j 0 As in simple linear regression, under the null hypothesis t 0 = ˆβ j ŝe( ˆβ j ) t n p 1. We reject H 0 if t 0 > t n p 1,1 α/2. This is a partial test because ˆβ j depends on all of the other predictors x i, i j that are in the model. Thus, this is a test of the contribution of x j given the other predictors in the model.
14 CHS example, cont. 13 y i = β 0 + weight i β 1 + height i β 2 + ɛ i H 0 : β 2 = 0 vs H 1 : β 2 0, given that weight is in the model. From the ANOVA table, ˆσ2 = C = (X X) 1 = t 0 = / = < t 495,.975 = 1.96 Therefore, we fail to reject the null hypothesis.
15 Tests for groups of predictors 14 Often it is of interest to determine whether a group of predictors contribute to predicting y given another predictor or group of predictors are in the model. In CHS example, we may want to know if age, height and sex are important predictors given weight is in the model when predicting blood pressure. We may want to know if additional powers of some predictor are important in the model given the linear term is already in the model. Given a predictor of interest, are interactions with other confounders of interest as well?
16 Using sums of squares to test for groups of predictors 15 Determine the contribution of a predictor or group of predictors to SSR given that the other regressors are in the model using the extrasumsofsquares method. Consider the regression model with p predictors y = Xβ + ɛ. We would like to determine if some subset of r < p predictors contributes significantly to the regression model.
17 Partition the vector of regression coefficients as β = [ ] β 1 β 2 16 where β 1 is (p + 1 r) 1 and β 2 is r 1. We want to test the hypothesis H 0 : β 2 = 0 Rewrite the model as where X = [X 1 X 2 ]. H 1 : β 2 0 y = Xβ + ɛ = X 1 β 1 + X 2 β 2 + ɛ, (1)
18 Equation (1) is the full model with SSR expressed as 17 SSR(X) = ˆβ X y (p+1 degrees of freedom) and MSE = y y ˆβ X y n p 1. To find the contribution of the predictors in X 2, fit the model assuming H 0 is true. This reduced model is y = X 1 β 1 + ɛ, where ˆβ 1 = (X 1 X 1 ) ( 1) X 1 y
19 and 18 SSR(X 1 ) = ˆβ 1 X 1 y (p+1r degrees of freedom). The regression sums of squares due to X 2 when X 1 is already in the model is SSR(X 2 X 1 ) = SSR(X) SSR(X 1 ) with r degrees of freedom. This is also known as the extra sum of squares due to X 2. SSR(X 2 X 1 ) is independent of MSE. We can test H 0 : β 2 = 0 with the statistic F 0 = SSR(X2 X 1 )/r MSE F r,n p 1.
20 CHS example, cont. 19 Full model: y i = β 0 + weight i β 1 + height i β 2 H 0 : β 2 = 0 Df Sum Sq Mean Sq F value Pr(>F) WEIGHT HEIGHT Residuals F 0 = / = 0.95 < F 1,495,0.95 = 3.86 This should look very similar to the ttest for H 0.
21 20 BP i = β 0 + weight i β 1 + height i β 2 + age i β 3 + gender i β 4 + ɛ > summary(lm(diabp~weight+height+age+gender,data=chs)) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e08 *** WEIGHT HEIGHT AGE *** GENDER Signif. codes: 0 *** ** 0.01 * Residual standard error: on 493 degrees of freedom Multiple RSquared: , Adjusted Rsquared: Fstatistic: on 4 and 493 DF, pvalue:
22 H 0 : β 2 = β 3 = β 4 = 0 vs H 1 : β j, j = 2, 3, 4 21 Df Sum Sq Mean Sq F value Pr(>F) WEIGHT HEIGHT AGE GENDER Residuals SSR(intercept, weight, height, age, gender) = = SSR(intercept, weight) = = SSR(height, age, gender intercept, weight) = = 1670 Notice we can also get this from the ANOVA table above SSR(height, age, gender intercept,weight) = = 1670
23 The observed F statistic is 22 F 0 = 1670/3/ = 13.5 > F 3,493,.95 = 2.62, and we reject the null hypothesis, concluding that at least one of β 2, β 3 or β 4 is not equal to 0. This should look very similar to the overall F test if we considered the intercept to be a predictor and all the covariates to be the additional variables under consideration.
24 What if we had put the predictors in the model in a different order? 23 diabp i = β 0 + height i β 2 + age i β 3 + weight i β 1 + gender i β 4 + ɛ Df Sum Sq Mean Sq F value Pr(>F) HEIGHT AGE WEIGHT GENDER Residuals Could we use this table to test H 0 : β 2 = β 3 = β 4 = 0?
25 What if we had the ANOVA table for the reduced model? Df Sum Sq Mean Sq F value Pr(>F) WEIGHT Residuals Given that SSR = SSR(x 2 ) + SSR(x 3 x 2 ) + SSR(x 1 x 2, x 3 ) + SSR(x 4 x 3, x 2, x 1 ) and then SSR(x 2, x 3, x 4 x 1 ) = SSR SSR(x 1 ) SSR(x 2, x 3, x 4 x 1 ) = = 1680.
26 One other question we might be interested in asking is if there are any significant interactions in the model? 25 lm(diabp~weight*height*age*gender,data=chs) Estimate Std. Error t value Pr(> t ) (Intercept) WEIGHT HEIGHT AGE GENDER WEIGHT:HEIGHT WEIGHT:AGE HEIGHT:AGE WEIGHT:GENDER HEIGHT:GENDER AGE:GENDER WEIGHT:HEIGHT:AGE WEIGHT:HEIGHT:GENDER WEIGHT:AGE:GENDER HEIGHT:AGE:GENDER WEIGHT:HEIGHT:AGE:GENDER
27 ANOVA table 26 Df Sum Sq Mean Sq F value Pr(>F) WEIGHT HEIGHT AGE GENDER WEIGHT:HEIGHT WEIGHT:AGE HEIGHT:AGE WEIGHT:GENDER HEIGHT:GENDER AGE:GENDER WEIGHT:HEIGHT:AGE WEIGHT:HEIGHT:GENDER WEIGHT:AGE:GENDER HEIGHT:AGE:GENDER WEIGHT:HEIGHT:AGE:GENDER Residuals
28 We can simplify the ANOVA table to 27 Df Sum Sq Mean Sq F value Pr(>F) Main effects Interactions Residuals How do we fill in the rest of this table?
121 Multiple Linear Regression Models
121.1 Introduction Many applications of regression analysis involve situations in which there are more than one regressor variable. A regression model that contains more than one regressor variable is
More informationMultiple Linear Regression
Multiple Linear Regression A regression with two or more explanatory variables is called a multiple regression. Rather than modeling the mean response as a straight line, as in simple regression, it is
More informationChapter 11: Linear Regression  Inference in Regression Analysis  Part 2
Chapter 11: Linear Regression  Inference in Regression Analysis  Part 2 Note: Whether we calculate confidence intervals or perform hypothesis tests we need the distribution of the statistic we will use.
More informationCov(x, y) V ar(x)v ar(y)
Simple linear regression Systematic components: β 0 + β 1 x i Stochastic component : error term ε Y i = β 0 + β 1 x i + ε i ; i = 1,..., n E(Y X) = β 0 + β 1 x the central parameter is the slope parameter
More informationRegression in ANOVA. James H. Steiger. Department of Psychology and Human Development Vanderbilt University
Regression in ANOVA James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) 1 / 30 Regression in ANOVA 1 Introduction 2 Basic Linear
More informationStatistics in Geophysics: Linear Regression II
Statistics in Geophysics: Linear Regression II Steffen Unkel Department of Statistics LudwigMaximiliansUniversity Munich, Germany Winter Term 2013/14 1/28 Model definition Suppose we have the following
More informationStat 411/511 ANOVA & REGRESSION. Charlotte Wickham. stat511.cwick.co.nz. Nov 31st 2015
Stat 411/511 ANOVA & REGRESSION Nov 31st 2015 Charlotte Wickham stat511.cwick.co.nz This week Today: Lack of fit Ftest Weds: Review email me topics, otherwise I ll go over some of last year s final exam
More informationChapter 11: Two Variable Regression Analysis
Department of Mathematics Izmir University of Economics Week 1415 20142015 In this chapter, we will focus on linear models and extend our analysis to relationships between variables, the definitions
More informationThe scatterplot indicates a positive linear relationship between waist size and body fat percentage:
STAT E150 Statistical Methods Multiple Regression Three percent of a man's body is essential fat, which is necessary for a healthy body. However, too much body fat can be dangerous. For men between the
More information0.1 Multiple Regression Models
0.1 Multiple Regression Models We will introduce the multiple Regression model as a mean of relating one numerical response variable y to two or more independent (or predictor variables. We will see different
More informationStatistics II Final Exam  January Use the University stationery to give your answers to the following questions.
Statistics II Final Exam  January 2012 Use the University stationery to give your answers to the following questions. Do not forget to write down your name and class group in each page. Indicate clearly
More informationRegression in SPSS. Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology
Regression in SPSS Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology John P. Bentley Department of Pharmacy Administration University of
More informationLecture 7 Linear Regression Diagnostics
Lecture 7 Linear Regression Diagnostics BIOST 515 January 27, 2004 BIOST 515, Lecture 6 Major assumptions 1. The relationship between the outcomes and the predictors is (approximately) linear. 2. The error
More informationMultivariate Analysis of Variance (MANOVA)
Multivariate Analysis of Variance (MANOVA) Example: (Spector, 987) describes a study of two drugs on human heart rate There are 24 subjects enrolled in the study which are assigned at random to one of
More informationWe extended the additive model in two variables to the interaction model by adding a third term to the equation.
Quadratic Models We extended the additive model in two variables to the interaction model by adding a third term to the equation. Similarly, we can extend the linear model in one variable to the quadratic
More informationInference in Regression Analysis
Yang Feng Inference in the Normal Error Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters X i is a known constant, the value of
More informationwhere b is the slope of the line and a is the intercept i.e. where the line cuts the y axis.
Least Squares Introduction We have mentioned that one should not always conclude that because two variables are correlated that one variable is causing the other to behave a certain way. However, sometimes
More informationStatistics for Management IISTAT 362Final Review
Statistics for Management IISTAT 362Final Review Multiple Choice Identify the letter of the choice that best completes the statement or answers the question. 1. The ability of an interval estimate to
More informationOutline. Topic 4  Analysis of Variance Approach to Regression. Partitioning Sums of Squares. Total Sum of Squares. Partitioning sums of squares
Topic 4  Analysis of Variance Approach to Regression Outline Partitioning sums of squares Degrees of freedom Expected mean squares General linear test  Fall 2013 R 2 and the coefficient of correlation
More informationRegression, least squares
Regression, least squares Joe Felsenstein Department of Genome Sciences and Department of Biology Regression, least squares p.1/24 Fitting a straight line X Two distinct cases: The X values are chosen
More informationStatistical Modelling in Stata 5: Linear Models
Statistical Modelling in Stata 5: Linear Models Mark Lunt Arthritis Research UK Centre for Excellence in Epidemiology University of Manchester 08/11/2016 Structure This Week What is a linear model? How
More informationMath 141. Lecture 24: Model Comparisons and The Ftest. Albyn Jones 1. 1 Library jones/courses/141
Math 141 Lecture 24: Model Comparisons and The Ftest Albyn Jones 1 1 Library 304 jones@reed.edu www.people.reed.edu/ jones/courses/141 Nested Models Two linear models are Nested if one (the restricted
More informationComparing Nested Models
Comparing Nested Models ST 430/514 Two models are nested if one model contains all the terms of the other, and at least one additional term. The larger model is the complete (or full) model, and the smaller
More informationRegression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur
Regression Analysis Prof. Soumen Maity Department of Mathematics Indian Institute of Technology, Kharagpur Lecture  7 Multiple Linear Regression (Contd.) This is my second lecture on Multiple Linear Regression
More informationData Mining and Data Warehousing. Henryk Maciejewski. Data Mining Predictive modelling: regression
Data Mining and Data Warehousing Henryk Maciejewski Data Mining Predictive modelling: regression Algorithms for Predictive Modelling Contents Regression Classification Auxiliary topics: Estimation of prediction
More informationSELFTEST: SIMPLE REGRESSION
ECO 22000 McRAE SELFTEST: SIMPLE REGRESSION Note: Those questions indicated with an (N) are unlikely to appear in this form on an inclass examination, but you should be able to describe the procedures
More informationEconometrics The Multiple Regression Model: Inference
Econometrics The Multiple Regression Model: João Valle e Azevedo Faculdade de Economia Universidade Nova de Lisboa Spring Semester João Valle e Azevedo (FEUNL) Econometrics Lisbon, March 2011 1 / 24 in
More informationRegression Analysis: Basic Concepts
The simple linear model Regression Analysis: Basic Concepts Allin Cottrell Represents the dependent variable, y i, as a linear function of one independent variable, x i, subject to a random disturbance
More informationInference in Regression Analysis. Dr. Frank Wood
Inference in Regression Analysis Dr. Frank Wood Inference in the Normal Error Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters
More informationTwoVariable Regression: Interval Estimation and Hypothesis Testing
TwoVariable Regression: Interval Estimation and Hypothesis Testing Jamie Monogan University of Georgia Intermediate Political Methodology Jamie Monogan (UGA) Confidence Intervals & Hypothesis Testing
More informationQuestions and Answers on Hypothesis Testing and Confidence Intervals
Questions and Answers on Hypothesis Testing and Confidence Intervals L. Magee Fall, 2008 1. Using 25 observations and 5 regressors, including the constant term, a researcher estimates a linear regression
More informationSimple Linear Regression
Inference for Regression Simple Linear Regression IPS Chapter 10.1 2009 W.H. Freeman and Company Objectives (IPS Chapter 10.1) Simple linear regression Statistical model for linear regression Estimating
More informationChapter 5: Linear regression
Chapter 5: Linear regression Last lecture: Ch 4............................................................ 2 Next: Ch 5................................................................. 3 Simple linear
More information5. Linear Regression
5. Linear Regression Outline.................................................................... 2 Simple linear regression 3 Linear model............................................................. 4
More informationttests and Ftests in regression
ttests and Ftests in regression Johan A. Elkink University College Dublin 5 April 2012 Johan A. Elkink (UCD) t and Ftests 5 April 2012 1 / 25 Outline 1 Simple linear regression Model Variance and R
More informationLinear Regression with One Regressor
Linear Regression with One Regressor Michael Ash Lecture 10 Analogy to the Mean True parameter µ Y β 0 and β 1 Meaning Central tendency Intercept and slope E(Y ) E(Y X ) = β 0 + β 1 X Data Y i (X i, Y
More informationLinear combinations of parameters
Linear combinations of parameters Suppose we want to test the hypothesis that two regression coefficients are equal, e.g. β 1 = β 2. This is equivalent to testing the following linear constraint (null
More informationBivariate Analysis. Correlation. Correlation. Pearson's Correlation Coefficient. Variable 1. Variable 2
Bivariate Analysis Variable 2 LEVELS >2 LEVELS COTIUOUS Correlation Used when you measure two continuous variables. Variable 2 2 LEVELS X 2 >2 LEVELS X 2 COTIUOUS ttest X 2 X 2 AOVA (Ftest) ttest AOVA
More informationName: Student ID#: Serial #:
STAT 22 Business Statistics II Term3 KING FAHD UNIVERSITY OF PETROLEUM & MINERALS Department Of Mathematics & Statistics DHAHRAN, SAUDI ARABIA STAT 22: BUSINESS STATISTICS II Third Exam July, 202 9:20
More informationRegression. Name: Class: Date: Multiple Choice Identify the choice that best completes the statement or answers the question.
Class: Date: Regression Multiple Choice Identify the choice that best completes the statement or answers the question. 1. Given the least squares regression line y8 = 5 2x: a. the relationship between
More informationPaired Differences and Regression
Paired Differences and Regression Students sometimes have difficulty distinguishing between paired data and independent samples when comparing two means. One can return to this topic after covering simple
More informationSchweser Printable Answers  Session Investment Tools: Quantitative Methods for Valuation
1 of 14 18/12/2006 6:42 Schweser Printable Answers  Session Investment Tools: Quantitative Methods for Valuation Test ID#: 1362402 Back to Test Review Hide Questions Print this Page Question 1  #12631
More informationData and Regression Analysis. Lecturer: Prof. Duane S. Boning. Rev 10
Data and Regression Analysis Lecturer: Prof. Duane S. Boning Rev 10 1 Agenda 1. Comparison of Treatments (One Variable) Analysis of Variance (ANOVA) 2. Multivariate Analysis of Variance Model forms 3.
More information(d) True or false? When the number of treatments a=9, the number of blocks b=10, and the other parameters r =10 and k=9, it is a BIBD design.
PhD Qualifying exam Methodology Jan 2014 Solutions 1. True or false question  only circle "true " or "false" (a) True or false? Fstatistic can be used for checking the equality of two population variances
More informationDEPARTMENT OF PSYCHOLOGY UNIVERSITY OF LANCASTER MSC IN PSYCHOLOGICAL RESEARCH METHODS ANALYSING AND INTERPRETING DATA 2 PART 1 WEEK 9
DEPARTMENT OF PSYCHOLOGY UNIVERSITY OF LANCASTER MSC IN PSYCHOLOGICAL RESEARCH METHODS ANALYSING AND INTERPRETING DATA 2 PART 1 WEEK 9 Analysis of covariance and multiple regression So far in this course,
More informationChapter 4: Constrained estimators and tests in the multiple linear regression model (Part II)
Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part II) Florian Pelgrin HEC SeptemberDecember 2010 Florian Pelgrin (HEC) Constrained estimators SeptemberDecember
More informationStatistical Models in R
Statistical Models in R Some Examples Steven Buechler Department of Mathematics 276B Hurley Hall; 16233 Fall, 2007 Outline Statistical Models Structure of models in R Model Assessment (Part IA) Anova
More informationANOVA. February 12, 2015
ANOVA February 12, 2015 1 ANOVA models Last time, we discussed the use of categorical variables in multivariate regression. Often, these are encoded as indicator columns in the design matrix. In [1]: %%R
More informationRegression Analysis. Pekka Tolonen
Regression Analysis Pekka Tolonen Outline of Topics Simple linear regression: the form and estimation Hypothesis testing and statistical significance Empirical application: the capital asset pricing model
More information15.1 The Regression Model: Analysis of Residuals
15.1 The Regression Model: Analysis of Residuals Tom Lewis Fall Term 2009 Tom Lewis () 15.1 The Regression Model: Analysis of Residuals Fall Term 2009 1 / 12 Outline 1 The regression model 2 Estimating
More informationIn Chapter 2, we used linear regression to describe linear relationships. The setting for this is a
Math 143 Inference on Regression 1 Review of Linear Regression In Chapter 2, we used linear regression to describe linear relationships. The setting for this is a bivariate data set (i.e., a list of cases/subjects
More informationCorrelation and Simple Linear Regression
Correlation and Simple Linear Regression We are often interested in studying the relationship among variables to determine whether they are associated with one another. When we think that changes in a
More informationWooldridge, Introductory Econometrics, 4th ed. Multiple regression analysis:
Wooldridge, Introductory Econometrics, 4th ed. Chapter 4: Inference Multiple regression analysis: We have discussed the conditions under which OLS estimators are unbiased, and derived the variances of
More informationResiduals. Residuals = ª Department of ISM, University of Alabama, ST 260, M23 Residuals & Minitab. ^ e i = y i  y i
A continuation of regression analysis Lesson Objectives Continue to build on regression analysis. Learn how residual plots help identify problems with the analysis. M231 M232 Example 1: continued Case
More informationSupplement 13A: Partial F Test
Supplement 13A: Partial F Test Purpose of the Partial F Test For a given regression model, could some of the predictors be eliminated without sacrificing too much in the way of fit? Conversely, would it
More informationOneWay Analysis of Variance: A Guide to Testing Differences Between Multiple Groups
OneWay Analysis of Variance: A Guide to Testing Differences Between Multiple Groups In analysis of variance, the main research question is whether the sample means are from different populations. The
More informationSPSS Guide: Regression Analysis
SPSS Guide: Regression Analysis I put this together to give you a stepbystep guide for replicating what we did in the computer lab. It should help you run the tests we covered. The best way to get familiar
More informationPractice 3 SPSS. Partially based on Notes from the University of Reading:
Practice 3 SPSS Partially based on Notes from the University of Reading: http://www.reading.ac.uk Simple Linear Regression A simple linear regression model is fitted when you want to investigate whether
More informationSoci708 Statistics for Sociologists
Soci708 Statistics for Sociologists Module 11 Multiple Regression 1 François Nielsen University of North Carolina Chapel Hill Fall 2009 1 Adapted from slides for the course Quantitative Methods in Sociology
More informationSUBMODELS (NESTED MODELS) AND ANALYSIS OF VARIANCE OF REGRESSION MODELS
1 SUBMODELS (NESTED MODELS) AND ANALYSIS OF VARIANCE OF REGRESSION MODELS We will assume we have data (x 1, y 1 ), (x 2, y 2 ),, (x n, y n ) and make the usual assumptions of independence and normality.
More information4.7 Confidence and Prediction Intervals
4.7 Confidence and Prediction Intervals Instead of conducting tests we could find confidence intervals for a regression coefficient, or a set of regression coefficient, or for the mean of the response
More informationStatistical Models in R
Statistical Models in R Some Examples Steven Buechler Department of Mathematics 276B Hurley Hall; 16233 Fall, 2007 Outline Statistical Models Linear Models in R Regression Regression analysis is the appropriate
More informationHow Do We Test Multiple Regression Coefficients?
How Do We Test Multiple Regression Coefficients? Suppose you have constructed a multiple linear regression model and you have a specific hypothesis to test which involves more than one regression coefficient.
More informationMultiple Regression in SPSS This example shows you how to perform multiple regression. The basic command is regression : linear.
Multiple Regression in SPSS This example shows you how to perform multiple regression. The basic command is regression : linear. In the main dialog box, input the dependent variable and several predictors.
More informationExam and Solution. Please discuss each problem on a separate sheet of paper, not just on a separate page!
Econometrics  Exam 1 Exam and Solution Please discuss each problem on a separate sheet of paper, not just on a separate page! Problem 1: (20 points A health economist plans to evaluate whether screening
More informationMultiple Hypothesis Testing: The Ftest
Multiple Hypothesis Testing: The Ftest Matt Blackwell December 3, 2008 1 A bit of review When moving into the matrix version of linear regression, it is easy to lose sight of the big picture and get lost
More information2SLS HATCO SPSS and SHAZAM Example. by Eddie Oczkowski. August X9: Usage Level (how much of the firm s total product is purchased from HATCO).
2SLS HATCO SPSS and SHAZAM Example by Eddie Oczkowski August 200 This example illustrates how to use SPSS to estimate and evaluate a 2SLS latent variable model. The bulk of the example relates to SPSS,
More informationE205 Final: Version B
Name: Class: Date: E205 Final: Version B Multiple Choice Identify the choice that best completes the statement or answers the question. 1. The owner of a local nightclub has recently surveyed a random
More informationChapter 16  Analyses of Variance and Covariance as General Linear Models Eye fixations per line of text for poor, average, and good readers:
Chapter 6  Analyses of Variance and Covariance as General Linear Models 6. Eye fixations per line of text for poor, average, and good readers: a. Design matrix, using only the first subject in each group:
More informationSection 3: Simple Linear Regression
Section 3: Simple Linear Regression Carlos M. Carvalho The University of Texas at Austin McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/ 1 Regression: General Introduction
More informationMultiple Linear Regression. Multiple linear regression is the extension of simple linear regression to the case of two or more independent variables.
1 Multiple Linear Regression Basic Concepts Multiple linear regression is the extension of simple linear regression to the case of two or more independent variables. In simple linear regression, we had
More informationIn this chapter, we aim to answer the following questions: 1. What is the nature of heteroskedasticity?
Lecture 9 Heteroskedasticity In this chapter, we aim to answer the following questions: 1. What is the nature of heteroskedasticity? 2. What are its consequences? 3. how does one detect it? 4. What are
More informationInference for Regression
Simple Linear Regression Inference for Regression The simple linear regression model Estimating regression parameters; Confidence intervals and significance tests for regression parameters Inference about
More informationChapter 7: Simple linear regression Learning Objectives
Chapter 7: Simple linear regression Learning Objectives Reading: Section 7.1 of OpenIntro Statistics Video: Correlation vs. causation, YouTube (2:19) Video: Intro to Linear Regression, YouTube (5:18) 
More informationHow to calculate an ANOVA table
How to calculate an ANOVA table Calculations by Hand We look at the following example: Let us say we measure the height of some plants under the effect of different fertilizers. Treatment Measures Mean
More informationEcon 371 Problem Set #3 Answer Sheet
Econ 371 Problem Set #3 Answer Sheet 4.1 In this question, you are told that a OLS regression analysis of third grade test scores as a function of class size yields the following estimated model. T estscore
More information7. Tests of association and Linear Regression
7. Tests of association and Linear Regression In this chapter we consider 1. Tests of Association for 2 qualitative variables. 2. Measures of the strength of linear association between 2 quantitative variables.
More informationStatistical Consulting Topics. MANOVA: Multivariate ANOVA
Statistical Consulting Topics MANOVA: Multivariate ANOVA Predictors are still factors, but we have more than one continuousvariable response on each experimental unit. For example, y i = (y i1, y i2 ).
More informationSCHOOL OF MATHEMATICS AND STATISTICS
RESTRICTED OPEN BOOK EXAMINATION (Not to be removed from the examination hall) Data provided: Statistics Tables by H.R. Neave MAS5052 SCHOOL OF MATHEMATICS AND STATISTICS Basic Statistics Spring Semester
More informationRegression III: Dummy Variable Regression
Regression III: Dummy Variable Regression Tom Ilvento FREC 408 Linear Regression Assumptions about the error term Mean of Probability Distribution of the Error term is zero Probability Distribution of
More informationFixed vs. Random Effects
Statistics 203: Introduction to Regression and Analysis of Variance Fixed vs. Random Effects Jonathan Taylor  p. 1/19 Today s class Implications for Random effects. Oneway random effects ANOVA. Twoway
More information1. What is the critical value for this 95% confidence interval? CV = z.025 = invnorm(0.025) = 1.96
1 Final Review 2 Review 2.1 CI 1propZint Scenario 1 A TV manufacturer claims in its warranty brochure that in the past not more than 10 percent of its TV sets needed any repair during the first two years
More informationNWay Analysis of Variance
NWay Analysis of Variance 1 Introduction A good example when to use a nway ANOVA is for a factorial design. A factorial design is an efficient way to conduct an experiment. Each observation has data
More informationBootstrapping Analogs of the One Way MANOVA Test
Bootstrapping Analogs of the One Way MANOVA Test Hasthika S Rupasinghe Arachchige Don and David J Olive Southern Illinois University March 17, 2016 Abstract The classical one way MANOVA model is used to
More informationWeighted Least Squares
Weighted Least Squares The standard linear model assumes that Var(ε i ) = σ 2 for i = 1,..., n. As we have seen, however, there are instances where Var(Y X = x i ) = Var(ε i ) = σ2 w i. Here w 1,..., w
More informationNotes on Applied Linear Regression
Notes on Applied Linear Regression Jamie DeCoster Department of Social Psychology Free University Amsterdam Van der Boechorststraat 1 1081 BT Amsterdam The Netherlands phone: +31 (0)20 4448935 email:
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANACHAMPAIGN
& ANOVA Edpsy 580 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANACHAMPAIGN Multivariate Relationships and Multiple Linear Regression Slide 1 of
More informationMultiple Regression Analysis in Minitab 1
Multiple Regression Analysis in Minitab 1 Suppose we are interested in how the exercise and body mass index affect the blood pressure. A random sample of 10 males 50 years of age is selected and their
More informationSection 13, Part 1 ANOVA. Analysis Of Variance
Section 13, Part 1 ANOVA Analysis Of Variance Course Overview So far in this course we ve covered: Descriptive statistics Summary statistics Tables and Graphs Probability Probability Rules Probability
More informationLinear constraints in multiple linear regression. Analysis of variance.
Section 16 Linear constraints in multiple linear regression. Analysis of variance. Multiple linear regression with general linear constraints. Let us consider a multiple linear regression Y = X + β and
More informationBiostatistics. ANOVA  Analysis of Variance. Burkhardt Seifert & Alois Tschopp. Biostatistics Unit University of Zurich
Biostatistics ANOVA  Analysis of Variance Burkhardt Seifert & Alois Tschopp Biostatistics Unit University of Zurich Master of Science in Medical Biology 1 ANOVA = Analysis of variance Analysis of variance
More informationPerform hypothesis testing
Multivariate hypothesis tests for fixed effects Testing homogeneity of level1 variances In the following sections, we use the model displayed in the figure below to illustrate the hypothesis tests. Partial
More informationTesting for Lack of Fit
Chapter 6 Testing for Lack of Fit How can we tell if a model fits the data? If the model is correct then ˆσ 2 should be an unbiased estimate of σ 2. If we have a model which is not complex enough to fit
More informationModule 5: Multiple Regression Analysis
Using Statistical Data Using to Make Statistical Decisions: Data Multiple to Make Regression Decisions Analysis Page 1 Module 5: Multiple Regression Analysis Tom Ilvento, University of Delaware, College
More informationSimple Linear Regression One Binary Categorical Independent Variable
Simple Linear Regression Does sex influence mean GCSE score? In order to answer the question posed above, we want to run a linear regression of sgcseptsnew against sgender, which is a binary categorical
More informationChapter 5 Analysis of variance SPSS Analysis of variance
Chapter 5 Analysis of variance SPSS Analysis of variance Data file used: gss.sav How to get there: Analyze Compare Means Oneway ANOVA To test the null hypothesis that several population means are equal,
More information4.4. Further Analysis within ANOVA
4.4. Further Analysis within ANOVA 1) Estimation of the effects Fixed effects model: α i = µ i µ is estimated by a i = ( x i x) if H 0 : µ 1 = µ 2 = = µ k is rejected. Random effects model: If H 0 : σa
More informationRegression Estimation  Least Squares and Maximum Likelihood. Dr. Frank Wood
Regression Estimation  Least Squares and Maximum Likelihood Dr. Frank Wood Least Squares Max(min)imization 1. Function to minimize w.r.t. β 0, β 1 Q = n (Y i (β 0 + β 1 X i )) 2 i=1 2. Minimize this by
More informationA. Karpinski
Chapter 3 Multiple Linear Regression Page 1. Overview of multiple regression 32 2. Considering relationships among variables 33 3. Extending the simple regression model to multiple predictors 34 4.
More informationPart II. Multiple Linear Regression
Part II Multiple Linear Regression 86 Chapter 7 Multiple Regression A multiple linear regression model is a linear model that describes how a yvariable relates to two or more xvariables (or transformations
More information