TRINITY COLLEGE. Faculty of Engineering, Mathematics and Science. School of Computer Science & Statistics

UNIVERSITY OF DUBLIN TRINITY COLLEGE Faculty of Engineering, Mathematics and Science School of Computer Science & Statistics BA (Mod) Enter Course Title Trinity Term 2013 Junior/Senior Sophister ST7002 Multiple Linear Regression Prof Haslett Date Venue Time Instructions to Candidates: Answer all questions.. All carry equal marks.in all questions, extra marks will be awarded for imaginative answers, including those that go beyond the question as posed when explaining and illustrating the ideas discussed Materials permitted for this examination: Calculator, Log tables Materials omitted from the front page of an examination paper will not be permitted during an examination Questions should start on page 2 only Page 1 of 13

Q1 Write short notes on EIGHT of the following topics. You should illustrate your notes by referring to examples. You may draw on other questions in this exam paper or on examples discussed in class. However additional credit will be given for the use of other examples. All topics carry equal marks a) Lessons from my project b) Objectives in regression c) Linear regression does not necessarily mean straight lines d) The role of the Normal distribution in regression e) Transformations f) Variance Inflation Factors g) Critical analysis of rows and columns/cases and variables h) Extra and Sequential Sums of Squares i) Sampling Distributions and Standard Errors in Regression j) Interactions Q2 In a clinical experiment, 64 patients (31/33, Male/Female) were administered a drug. The response (mg/l, based on a blood analysis 12 hours later) was noted. Three drug levels were used (200,400,600); the gender and weights (kg) of the patients were noted. (Naturally, the average Male/Female weights differed.). Several analyses are reported as below. a) In a pair of preliminary analyses, the response by gender was analysed, as overleaf (Q2A) Explain how these two analyses should be interpreted. (6 marks) b) Two further simple analyses are reported (Q2B) Interpret the analyses. Explain the different interpretations of the Confidence and Prediction intervals. (6 marks) Page 2 of 13

resp resp resp XST7002 Q2A Two-Sample T-Test: resp, Gender Gender N Mean StDev SE Mean 0 33 1.611 0.604 0.11 1 31 1.244 0.566 0.10 Regression of Response on Gender. M=1; F=0 Difference = mu (0) - mu (1) Estimate for difference: 0.367 95% CI for difference: (0.074, 0.660) T-Test of difference = 0 (vs not =): T-Value = 2.51 P-Value = 0.015 DF =62 2.5 Resp vs Gender M,F/1,0 resp = 1.611-0.3673 Gender S 0.585914 R-Sq 9.2% R-Sq(adj) 7.7% 2.0 Regression Analysis: resp versus Gender 1.5 1.0 resp = 1.61-0.367 Gender 0.5 0.0 0.0 0.2 0.4 0.6 Gender 0.8 1.0 Predictor Coef SE Coef T P Constant 1.6109 0.1020 15.79 0.000 Gender -0.3673 0.1466-2.51 0.015 S = 0.585914 R-Sq = 9.2 Q2B Regression Analysis: resp versus dose resp = - 0.015 + 0.00368 dose Predictor Coef SE Coef T P Constant -0.0146 0.11680-0.12 0.901 dose 0.00367 0.00028 13.16 0.000 S = 0.315622 R-Sq = 73.7% Regression Analysis: resp versus wt resp = 3.556-0.03832 wt Predictor Coef SE Coef T P Constant 3.5563 0.5747 6.19 0.000 wt -0.0383 0.01030-3.72 0.000 S = 0.555914 R-Sq = 18.3% 2.5 2.0 Resp vs Dose resp = - 0.0146 + 0.003676 dose Regression 95% CI 95% PI S 0.315622 R-Sq 73.7% R-Sq(adj) 73.2% 3.5 2.5 2.0 Resp vs Wt resp = 3.556-0.03832 wt Regression 95% CI 95% PI S 0.555914 R-Sq 18.3% R-Sq(adj) 16.9% 1.5 1.5 1.0 0.5 1.0 0.5 0.0 0.0 200 300 400 dose 500 600-0.5 40 45 50 55 wt 60 65 70 75 Q2 Continues Page 3 of 13

c) Two multiple regression analyses are presented below Q2C The researcher is puzzled by their different interpretations as regards the apparent importance of dosage. The dose/wt variable is a derived variable, being the ratio of dose to weight. Provide her with an explanation. Use this to discuss ideas of correlated predictor variables and of direct and indirect relationships. d) A further derived variable, (Gender*dose/wt) is created; this is the simple (8 marks) product of the binary Gender variable by dose/wt. The resulting regression is in Q2D. What is the purpose of including such a variable in such a regression analysis? What is the interpretation in this case? Contrast with the analysis above. Illustrate your discussion rough sketches of the two simple regression lines (resp vs dose/wt) that are implicit in this model. (8 marks) e) A final analysis leads to Q2E. Discuss the interpretation. Would you propose further analyses? Q2C Regression Analysis: resp versus wt, Gender, dose resp = 2.35-0.0447 wt + 0.222 Gender + 0.00369 dose Predictor Coef SE Coef T P Constant 2.3494 0.2637 8.91 0.000 wt -0.044693 0.005023-8.90 0.000 Gender 0.22194 0.06882 3.23 0.002 dose 0.0036889 0.0001815 20.33 0.000 S = 0.201128 R-Sq = 89.6% Regression Analysis: resp versus wt, Gender, dose, dose/wt resp = 0.937-0.0195 wt + 0.210 Gender + 0.00062 dose + 0.170 dose/wt Predictor Coef SE Coef T P VIF Constant 0.9372 0.6207 1.51 0.136 wt -0.01955 0.01118-1.75 0.086 9.795 Gender 0.21016 0.06618 3.18 0.002 1.881 dose 0.000621 0.001244 0.50 0.620 597 dose/wt 0.17021 0.06832 2.49 0.016 62.853 (5 marks) S = 0.192930 R-Sq = 90.6% Q2 Continues Page 4 of 13

Q2D Regression Analysis: resp versus Gender, dose/wt, Gender*dose/wt resp = - 0.130 + 0.089 Gender + 0.210 dose/wt - 0.0000 Gender*dose/wt Predictor Coef SE Coef T P Constant -0.1301 0.1162-1.12 0.267 Gender 0.0887 0.1525 0.58 0.563 dose/wt 0.20986 0.01333 15.75 0.000 Gender*dose/wt -0.00001 0.02004-0.00 1.000 S = 0.205411 R-Sq = 89.2% R-Sq(adj) = 88.7% Q2E Regression Analysis: resp versus dose/wt, Gender, wt, dose, Gender*dose/ resp = 0.808 + 0.189 dose/wt + 0.151 Gender - 0.0166 wt + 0.00021 dose + 0.0086 Gender*dose/wt Predictor Coef SE Coef T P VIF Constant 0.8077 0.7636 1.06 0.295 dose/wt 0.18872 0.09303 2.03 0.047 114.757 Gender 0.1509 0.2112 0.71 0.478 18.867 wt -0.01663 0.01497-1.11 0.271 17.277 dose 0.000207 0.001876 0.11 0.912 118.972 Gender*dose/wt 0.00856 0.02892 0.30 0.768 17.424 S = 0.194440 R-Sq = 90.6% R-Sq(adj) = 89.8% Analysis of Variance Source DF SS MS F P Regression 5 21.2485 4.2497 112.41 0.000 Residual Error 58 2.1928 0.0378 Total 63 23.4413 Source DF Seq SS dose/wt 1 20.8030 Gender 1 0.1067 wt 1 0.3262 dose 1 0.0093 Gender*dose/wt 1 0.0033 Page 5 of 13

Q3 Data are available on the Weight (gms) and physical dimensions Length, Width and Height (cms) of 56 perch. All are caught from the same lake (Laengelmavesi) near Tampere in Finland. A matrix plot and various analyses are presented below. The interest lies in relating the dimensions to the weight. Matrix Plot of Weight, Length, Ht, width 10 30 50 3 6 9 1000 Weight 500 50 0 30 Length3 10 12 Ht 8 4 9 6 3 width 0 500 1000 4 8 12 a) It is immediately apparent that separate linear regressions of Weight on Length, Width and Height will encounter difficulties. Discuss. (6 marks) b) All variables were log transformed; the resulting multiple regression analysis as in Q3A overleaf. Discuss the various aspects of this transformation and subsequent analysis. What are the implications of the VIF values? (6 marks) c) In an attempt at a simpler model, the derived variable Vol=Length Height Width was formed. The Fitted Line plot in C overleaf summarises the analysis; the SE for the slope is returned as 0.011. Explain the features of this analysis and plot. (8 marks) d) Use the models in (b) and (c) above, to compute approximate 95% Prediction Intervals for the Weights of two fish with dimensions (Length, Height and Width) being respectively (14.7, 3.5 and 2.0) and (45.2, 11.9 and 7.3). Explain carefully the basis, in the fitted models, for your calculations. (8 marks) e) The slope SE is 0.011. What are the implications for possible further simplification? (3 marks) f) It is remarked that although the last model (c) provides an excellent and simple fit, its interpretation differs to an extent that seems to be statistically significant - from the details of the model fitted in (b). Discuss. (2 marks) Page 6 of 13

Frequency Deleted Residual Percent Deleted Residual XST7002 Q3A Regression Analysis: logwt versus loglen, loght, logwidth logwt = - 1.07 + 1.65 loglen + 0.805 loght + 0.555 logwidth Predictor Coef SE Coef T P VIF Constant -1.0745 0.1738-6.18 0.000 loglen 1.6519 0.2250 7.34 0.000 44.835 loght 0.8053 0.2143 3.76 0.000 51.544 logwidth 0.5552 0.1780 3.12 0.003 35.799 S = 0.0371767 R-Sq = 99.4% R-Sq(adj) = 99.4% Unusual Observations Obs loglen logwt Fit SE Fit Residual St Resid 1 0.94 0.77085 0.82968 0.02001-0.05883-1.88 X 6 1.28 2.00000 1.91295 0.01186 0.08705 2.47R 19 1.37 2.04139 2.12198 0.01449-0.08059-2.35R 23 1.39 2.17609 2.10871 0.01715 0.06738 2.04R 25 1.41 2.35218 2.26083 0.01361 0.09135 2.64R 40 1.57 2.92428 2.87080 0.01820 0.05348 1.65 X Q3B R denotes an observation with a large standardized residual. X denotes an observation whose X value gives it large leverage. 99 Normal Probability Plot Residual Plots for logwt Versus Fits 90 1.5 50 0.0 10-1.5 1 - -1.5 0.0 Deleted Residual 1.5-1.0 1.5 2.0 Fitted Value 2.5 16 Histogram Versus Order 12 1.5 8 0.0 4-1.5 Q3C 0-2.4-1.2 0.0 1.2 Deleted Residual 2.4-1 5 10 15 20 25 30 35 40 Observation Order 45 50 55 Page 7 of 13

Weight XST7002 1600 1400 1200 1000 Weight vs Vol log10(weight) = - 0.5230 + 0.9785 log10(vol) Regression 95% PI S 0.0401304 R-Sq 99.3% R-Sq(adj) 99.3% 800 600 400 200 0 0 1000 2000 Vol 3000 4000 5000 Page 8 of 13

Outline Solution Q2 a) The two analyses are equivalent, though differently packaged. The T-test reports that the observed means and mean difference in response are 1.611, 1.244 and 0.367. The regression reports the same info (to within rounding error): when Gender =0 (F) the regression reports the expected value to be 1.61; when Gender =1 (M) the regression reports 1.61-0.367 = 1.243. The regression model regards Gender and as an Indicator variable; the plot which interpolates to other values of Gender is not interpretable. Treating the remaining variation (other than due to Gender) can be regarded as random, both find that the T-ratio for the difference is 2.51. Both report that this is statistically significant. b) Resp vs dose reports in increase in the average increase in response of 0.0037 per unit of dose; this is 0.0037 200=0.74 per 200 units. If the remaining error can be treated as random, this is hugely statistically significant. Res vs wt shows an apparent reduction in response associated with weight, also statistically significant. Anticipating later analyses, response is often more naturally sensitive to dose per unit weight; the latter analysis is consistent with this. The prediction intervals as shown are effectively descriptive. Most of the data lie within these. The Confidence Intervals qualify statements about the mean response (over very many patients with specified dose or weight. One interpretation is that regression lines that are statistically consistent with the data must lie within the CI band. Page 9 of 13

c) The first analysis suggests that weight, dose and gender are all important predictors of response. The second suggests that, when dose/wt is included as a predictor, neither dose nor weight contribute much Dose extra information. We already know that weight and gender and interrelated. And by construction Dose/wt Weight does/wt is correlated to dose and Gender to weight. The apparent confusion arises frequently when the x-variable (predictor variables) are themselves inter-related. The slope coefficients are in such cases - not simply interpretable in terms of the corresponding bivariate correlations. Resp The diagram not a requirement, but worthy of marks if offered provides one way to envision a possible set of direct and indirect relationships with response. The concept of direct and indirect relationships has been discussed in class. d) The new derived variable simplifies the direct consideration and comparison of two simple models: Resp vs dose/wt separately for M/F. Separately these may be written as resp = int cpt + slope (dose/wt), with potentially different values for each for M/F. This can also be a way to investigate interaction. Here F -0.1301 + 0.20986 dose/wt M (-0.1301+0.0887) + (0.20986-0.00001) dose/wt Lines, when sketched, are effectively parallel and have almost same slope). Since the values +0.0887 and -0.00001 are small compared to SE, via T = 0.58, T = 0.00 we can conclude that a single regression relationship, for both M and F, is likely to be adequate. This in turn suggest that the Weight and Gender terms in the second model in c) may not be simply interpreted as suggested there as individually necessary. For the Gender term is correlated with Weight. Perhaps the inclusion of Weight requires the inclusion of Gender to counter-balance it. e) The analysis confirms that the important variable is dose/wt. No other variables are significant. However, the very high VIF values for dose and dose/wt point to the fact that these variables are (naturally) highly interdependent. One of these is likely to be to most important. The choice should be guided by the considerations of the way the drug interacts, biochemically, with the patient. Page 10 of 13

Q3 a) It is clear that the bivariate relationships (top row) are not linear. Additionally, there is clear evidence of variance of weight increasing with weight. It is also the case that there is a great deal of correlation between the 3 x variables, likely to cause problems if they are ever used together. b) The model in the log scale shows that all vars are very significantly different from 0; tho that was never in doubt. R 2 is high. This can also be written as Wt = Len 1.65 Ht 0.81 Width 0.56 10 error. The nominal interpretation is that an increase of 1 in eg loglen ( ie an increase of 10- in Len) will induce - on average an increase of 1.65 in logwt (ie 10 1.65 =45-fold in Wt) if all other variables are held constant. But The VIF values suggest that the covariates are correlated- as anticipated and that the SEs are therefore inflated. This was apparent also in the scatterplots. Effectively this means that some fish are large in respect of all three dimensions, and some are small. In these circumstances one option is to choose a single composite that reflects all of the variables. It is likely to be futile to choose one of them. There are however a number of unusual observations. 4 of these exhibit large residuals, which merit attention. Three are very large and positive, and one is large and negative. Two are influential, being far from the others in respect of (log)len, Ht, Width. There is nothing wrong with this, necessarily. c) The option followed was to choose a product named Vol. The Fitted Line plot has fitted Weight to Vol, in the log scale, presenting the analysis in the back-transformed anti-log scale. Alternatively as Log(Vol) = Log(Length)+ Log(Height) + Log(Width), the composite Log(Vol) variable is the sum of the three Log(covariates). The fitted model can be written as Wt = 10-0.052 Vol 0.98 10 error = 0.3 Vol 0.98 10 error = 0.3 Len 0.98 Ht 0.98 Width 0.98 10 error. An increase of 1 in LogVol (ie a 10-fold increase) will generate, on average a 10 0.98 -fold (ie 9.55-fold) increase in Wt, on average. The constant 0.3 could be thought of as fish-density, were fish to be cuboid. As it is, it is a combination of fish density and the ratio of actual fish volume to the volume of the corresponding cuboid. When Vol 0.98 is large (ie when Wt is large) the absolute errors implicit in a 10 error -fold variation are large.this exhibits the fan-like figure for the prediction intervals Predicting the weight if a large fish is harder than that of a small fish in absolute terms. The issue is equivalent to describing the prediction error in %age terms. Page 11 of 13

The R2 value is almost as high as in b). The value of s is 0.04. This is in the logscale and compares with the value of 0.037 above. The model is not quite as tight-fitting, but it is simpler. No info is available on unusual obs, or SEs. d) The two prediction equations are: (as in b) Pred logwt = -1.07 + 1.65 Log(len)+ 0.805 log(ht)m + 0.555 Log(Width) 2(0.037) (as in c) Pred logwt = -0.523 + 0.9785 Log(len*Ht*Width) 2(0.040), back transformed as antilog( Pred log(wt)) as below Fish dimensions Fish len ht width Vol 1 14.7 3.5 2 102.9 2 45.2 11.9 7.3 3926.5 log 10 dimensions Log(vol) sum log 1 1.167 0.544 0.301 2.012 2.012 2 1.655 1.076 0.863 3.594 3.594 model b coeffs model c coeff const len ht width s const vol s -1.07 1.65 0.805 0.555 0.037-0.523 0.9785 0.04 Fish Pred LogWt lo hi Pred LogWt lo hi 1 1.46 1.39 1.54 1.45 1.37 1.53 2 1 2.93 8 2.99 2.91 7 backtransform backtransform 1 29 24 34 28 23 34 2 1014 855 1202 986 820 1185 Conclusions: very similar See f) below Model c has a slope is 0.9785 2 ( 0.011). This includes coeff=1. That is, the data are statistically consistent with 1; a Null Hyp: slope =1 would not be rejected. A simpler version of this model would then be LogWt = const + Log Vol. Equiv this is Wt=0.3 Vol. This model is not unlike the Tree model discussed in class. Note that the SE (0.011) is very much smaller than the SE s for each of the dimensions in model b). That s because their SE s have been inflated by, effectively, the lack of determinancy of the separate coeffs. Note however, that the correlation between these dimensions has no implications at all for the usefulness of the prediction equation generated by model b). It is simply the case that many different combinations of these coefficients are effectively equivalent to each other. e) However, model (c) corresponds to giving coefficients of 1 to each (log) dimension. This is just about consistent with the fit for LogHt (0.805 2(0.21) it is not as consistent with LogLen (1.65 2(0.22) and LogWidth (0.55 2(0.18). The implications are that Fish that are very long will be given low values of log Wt in model c and Fish that are very wide will be given high values of Page 12 of 13

LogWt in model c. Fish are not cuboids. However it is moot whether the data would require a rejection (in model b) of the Null Hyp that all coefficients were equal to 1. XST7002 Page 13 of 13