We extended the additive model in two variables to the interaction model by adding a third term to the equation.

Transcription

1 Quadratic Models We extended the additive model in two variables to the interaction model by adding a third term to the equation. Similarly, we can extend the linear model in one variable to the quadratic model by adding a second term to the equation: E(Y ) = β 0 + β 1 x + β 2 x 2. This a special case of the two-variable model with x 1 = x and x 2 = x 2. E(Y ) = β 0 + β 1 x 1 + β 2 x 2 1 / 16 Multiple Linear Regression Quadratic Models

2 Example: immune system and exercise x = maximal oxygen uptake (VO 2 max, ml/(kg min)); y = immunoglobulin level (IgG, mg/dl); data for 30 subjects (AEROBIC.txt). Get the data and plot them: aerobic <- read.table("text/exercises&examples/aerobic.txt", header = TRUE) plot(aerobic[, c("maxoxy", "IGG")]) Slight curvature suggests a linear model may not fit. 2 / 16 Multiple Linear Regression Quadratic Models

3 Check the linear model: plot(lm(igg ~ MAXOXY, aerobic)) Graph of residuals against fitted values shows definite curvature. Fit and summarize the quadratic model: aerobiclm <- lm(igg ~ MAXOXY + I(MAXOXY^2), aerobic) summary(aerobiclm) 3 / 16 Multiple Linear Regression Quadratic Models

4 Output Call: lm(formula = IGG ~ MAXOXY + I(MAXOXY^2), data = aerobic) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) ** MAXOXY e-05 *** I(MAXOXY^2) ** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 27 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 2 and 27 DF, p-value: < 2.2e-16 4 / 16 Multiple Linear Regression Quadratic Models

5 The quadratic term I(MAXOXY^2) is significant, so we reject the null hypothesis that the linear model is acceptable. The quadratic term is negative, which is consistent with the concavity of the curve. The other two t-ratios test irrelevant hypotheses, because the quadratic term is important. Extrapolation: the fitted curve has a maximum at MAXOXY = and declines for higher MAXOXY, which seems unlikely to represent the real relationship. 5 / 16 Multiple Linear Regression Quadratic Models

6 An alternative analysis The graph of IGG against log(maxoxy) is more linear: with(aerobic, plot(log(maxoxy), IGG)) aerobiclm2 <- lm(igg ~ log(maxoxy), aerobic) summary(aerobiclm2) with(aerobic, plot(maxoxy, IGG)) with(aerobic, lines(sort(maxoxy), fitted(aerobiclm)[order(maxoxy)], col = "blue")) with(aerobic, lines(sort(maxoxy), fitted(aerobiclm2)[order(maxoxy)], col = "red")) The fitted curve continues to increase indefinitely, but with diminishing slope. 6 / 16 Multiple Linear Regression Quadratic Models

7 Output Call: lm(formula = IGG ~ log(maxoxy), data = aerobic) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-15 *** log(maxoxy) < 2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 28 degrees of freedom Multiple R-squared: 0.934, Adjusted R-squared: F-statistic: on 1 and 28 DF, p-value: < 2.2e-16 7 / 16 Multiple Linear Regression Quadratic Models

8 More Complex Models ST 430/514 Complete second-order model When the first-order model E(Y ) = β 0 + β 1 x 1 + β 2 x 2 is inadequate, the interaction model E(Y ) = β 0 + β 1 x 1 + β 2 x 2 + β 3 x 1 x 2 may be better, but sometimes a complete second-order model is needed: E(Y ) = β 0 + β 1 x 1 + β 2 x 2 + β 3 x 1 x 2 + β 4 x β 5 x / 16 Multiple Linear Regression More Complex Models

9 Example: cost of shipping packages Get the data and plot them: express <- read.table("text/exercises&examples/express.txt", header = TRUE) pairs(express) Fit the complete second-order model and summarize it: expresslm <- lm(cost ~ Weight * Distance + I(Weight^2) + I(Distance^2), express) summary(expresslm) plot(expresslm) 9 / 16 Multiple Linear Regression More Complex Models

10 Output ST 430/514 Call: lm(formula = Cost ~ Weight * Distance + I(Weight^2) + I(Distance^2), data = express) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) 8.270e e Weight e e ** Distance 4.021e e I(Weight^2) 8.975e e *** I(Distance^2) 1.507e e Weight:Distance 7.327e e e-08 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 14 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 5 and 14 DF, p-value: 5.371e / 16 Multiple Linear Regression More Complex Models

11 Qualitative Variables A qualitative variable (or factor) is one that indicates membership of different categories. E.g., a person s gender = male or female: a qualitative variable with two levels, indicating membership of one of two categories. E.g., package type = Fragile, Semifragile, or Durable: three levels, corresponding to three categories. 11 / 16 Multiple Linear Regression More Complex Models

12 We code a qualitative variable using indicator (dummy) variables: Choose one level to use as a base or reference level, say male or Durable. For each other level, create a variable { 1 if this item is in this category x j = 0 otherwise. For gender, there is only one other category, so the only indicator variable is { 1 for a female x = 0 for a male. 12 / 16 Multiple Linear Regression More Complex Models

13 For packages, there are two other categories, so the indicator variables are { 1 for a Fragile package x Fragile = 0 otherwise, { 1 for a Semifragile package x Semifragile = 0 otherwise, For any item, at most one of the indicator variables is non-zero, indicating a non-base category; if they are all zero, the item belongs to the base category. 13 / 16 Multiple Linear Regression More Complex Models

14 Example: shipment cost of packages, by type. Get the data and plot them: cargo <- read.table("text/exercises&examples/cargo.txt", header = TRUE) plot(cost ~ CARGO, cargo) Fit and summarize the model: cargolm <- lm(cost ~ CARGO, cargo) summary(cargolm) 14 / 16 Multiple Linear Regression More Complex Models

15 Output Call: lm(formula = COST ~ CARGO, data = cargo) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) * CARGOFragile e-05 *** CARGOSemiFrag ** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 12 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 2 and 12 DF, p-value: / 16 Multiple Linear Regression More Complex Models

16 Note that the intercept is the fitted value for CARGOFragile = 0 and CARGOSemiFrag = 0; that is, for Durable packages. The coefficients of CARGOFragile and CARGOSemiFrag measure the differences between those categories and Durable. The overall model F -test is the same as the analysis of variance test: cargoaov <- aov(cost ~ CARGO, cargo) summary(cargoaov) Output Df Sum Sq Mean Sq F value Pr(>F) CARGO *** Residuals Signif. codes: 0 *** ** 0.01 * / 16 Multiple Linear Regression More Complex Models