TRINITY COLLEGE. Faculty of Engineering, Mathematics and Science. School of Computer Science & Statistics

Transcription

1 UNIVERSITY OF DUBLIN TRINITY COLLEGE Faculty of Engineering, Mathematics and Science School of Computer Science & Statistics BA (Mod) Enter Course Title Trinity Term 2013 Junior/Senior Sophister ST7002 Multiple Linear Regression Prof Haslett Date Venue Time Instructions to Candidates: Answer all questions.. All carry equal marks.in all questions, extra marks will be awarded for imaginative answers, including those that go beyond the question as posed when explaining and illustrating the ideas discussed Materials permitted for this examination: Calculator, Log tables Materials omitted from the front page of an examination paper will not be permitted during an examination Questions should start on page 2 only Page 1 of 13

2 Q1 Write short notes on EIGHT of the following topics. You should illustrate your notes by referring to examples. You may draw on other questions in this exam paper or on examples discussed in class. However additional credit will be given for the use of other examples. All topics carry equal marks a) Lessons from my project b) Objectives in regression c) Linear regression does not necessarily mean straight lines d) The role of the Normal distribution in regression e) Transformations f) Variance Inflation Factors g) Critical analysis of rows and columns/cases and variables h) Extra and Sequential Sums of Squares i) Sampling Distributions and Standard Errors in Regression j) Interactions Q2 In a clinical experiment, 64 patients (31/33, Male/Female) were administered a drug. The response (mg/l, based on a blood analysis 12 hours later) was noted. Three drug levels were used (200,400,600); the gender and weights (kg) of the patients were noted. (Naturally, the average Male/Female weights differed.). Several analyses are reported as below. a) In a pair of preliminary analyses, the response by gender was analysed, as overleaf (Q2A) Explain how these two analyses should be interpreted. (6 marks) b) Two further simple analyses are reported (Q2B) Interpret the analyses. Explain the different interpretations of the Confidence and Prediction intervals. (6 marks) Page 2 of 13

3 resp resp resp XST7002 Q2A Two-Sample T-Test: resp, Gender Gender N Mean StDev SE Mean Regression of Response on Gender. M=1; F=0 Difference = mu (0) - mu (1) Estimate for difference: % CI for difference: (0.074, 0.660) T-Test of difference = 0 (vs not =): T-Value = 2.51 P-Value = DF = Resp vs Gender M,F/1,0 resp = Gender S R-Sq 9.2% R-Sq(adj) 7.7% 2.0 Regression Analysis: resp versus Gender resp = Gender Gender Predictor Coef SE Coef T P Constant Gender S = R-Sq = 9.2 Q2B Regression Analysis: resp versus dose resp = dose Predictor Coef SE Coef T P Constant dose S = R-Sq = 73.7% Regression Analysis: resp versus wt resp = wt Predictor Coef SE Coef T P Constant wt S = R-Sq = 18.3% Resp vs Dose resp = dose Regression 95% CI 95% PI S R-Sq 73.7% R-Sq(adj) 73.2% Resp vs Wt resp = wt Regression 95% CI 95% PI S R-Sq 18.3% R-Sq(adj) 16.9% dose wt Q2 Continues Page 3 of 13

4 c) Two multiple regression analyses are presented below Q2C The researcher is puzzled by their different interpretations as regards the apparent importance of dosage. The dose/wt variable is a derived variable, being the ratio of dose to weight. Provide her with an explanation. Use this to discuss ideas of correlated predictor variables and of direct and indirect relationships. d) A further derived variable, (Gender*dose/wt) is created; this is the simple (8 marks) product of the binary Gender variable by dose/wt. The resulting regression is in Q2D. What is the purpose of including such a variable in such a regression analysis? What is the interpretation in this case? Contrast with the analysis above. Illustrate your discussion rough sketches of the two simple regression lines (resp vs dose/wt) that are implicit in this model. (8 marks) e) A final analysis leads to Q2E. Discuss the interpretation. Would you propose further analyses? Q2C Regression Analysis: resp versus wt, Gender, dose resp = wt Gender dose Predictor Coef SE Coef T P Constant wt Gender dose S = R-Sq = 89.6% Regression Analysis: resp versus wt, Gender, dose, dose/wt resp = wt Gender dose dose/wt Predictor Coef SE Coef T P VIF Constant wt Gender dose dose/wt (5 marks) S = R-Sq = 90.6% Q2 Continues Page 4 of 13

5 Q2D Regression Analysis: resp versus Gender, dose/wt, Gender*dose/wt resp = Gender dose/wt Gender*dose/wt Predictor Coef SE Coef T P Constant Gender dose/wt Gender*dose/wt S = R-Sq = 89.2% R-Sq(adj) = 88.7% Q2E Regression Analysis: resp versus dose/wt, Gender, wt, dose, Gender*dose/ resp = dose/wt Gender wt dose Gender*dose/wt Predictor Coef SE Coef T P VIF Constant dose/wt Gender wt dose Gender*dose/wt S = R-Sq = 90.6% R-Sq(adj) = 89.8% Analysis of Variance Source DF SS MS F P Regression Residual Error Total Source DF Seq SS dose/wt Gender wt dose Gender*dose/wt Page 5 of 13

6 Q3 Data are available on the Weight (gms) and physical dimensions Length, Width and Height (cms) of 56 perch. All are caught from the same lake (Laengelmavesi) near Tampere in Finland. A matrix plot and various analyses are presented below. The interest lies in relating the dimensions to the weight. Matrix Plot of Weight, Length, Ht, width Weight Length Ht width a) It is immediately apparent that separate linear regressions of Weight on Length, Width and Height will encounter difficulties. Discuss. (6 marks) b) All variables were log transformed; the resulting multiple regression analysis as in Q3A overleaf. Discuss the various aspects of this transformation and subsequent analysis. What are the implications of the VIF values? (6 marks) c) In an attempt at a simpler model, the derived variable Vol=Length Height Width was formed. The Fitted Line plot in C overleaf summarises the analysis; the SE for the slope is returned as Explain the features of this analysis and plot. (8 marks) d) Use the models in (b) and (c) above, to compute approximate 95% Prediction Intervals for the Weights of two fish with dimensions (Length, Height and Width) being respectively (14.7, 3.5 and 2.0) and (45.2, 11.9 and 7.3). Explain carefully the basis, in the fitted models, for your calculations. (8 marks) e) The slope SE is What are the implications for possible further simplification? (3 marks) f) It is remarked that although the last model (c) provides an excellent and simple fit, its interpretation differs to an extent that seems to be statistically significant - from the details of the model fitted in (b). Discuss. (2 marks) Page 6 of 13

7 Frequency Deleted Residual Percent Deleted Residual XST7002 Q3A Regression Analysis: logwt versus loglen, loght, logwidth logwt = loglen loght logwidth Predictor Coef SE Coef T P VIF Constant loglen loght logwidth S = R-Sq = 99.4% R-Sq(adj) = 99.4% Unusual Observations Obs loglen logwt Fit SE Fit Residual St Resid X R R R R X Q3B R denotes an observation with a large standardized residual. X denotes an observation whose X value gives it large leverage. 99 Normal Probability Plot Residual Plots for logwt Versus Fits Deleted Residual Fitted Value Histogram Versus Order Q3C Deleted Residual Observation Order Page 7 of 13

8 Weight XST Weight vs Vol log10(weight) = log10(vol) Regression 95% PI S R-Sq 99.3% R-Sq(adj) 99.3% Vol Page 8 of 13

9 Outline Solution Q2 a) The two analyses are equivalent, though differently packaged. The T-test reports that the observed means and mean difference in response are 1.611, and The regression reports the same info (to within rounding error): when Gender =0 (F) the regression reports the expected value to be 1.61; when Gender =1 (M) the regression reports = The regression model regards Gender and as an Indicator variable; the plot which interpolates to other values of Gender is not interpretable. Treating the remaining variation (other than due to Gender) can be regarded as random, both find that the T-ratio for the difference is Both report that this is statistically significant. b) Resp vs dose reports in increase in the average increase in response of per unit of dose; this is =0.74 per 200 units. If the remaining error can be treated as random, this is hugely statistically significant. Res vs wt shows an apparent reduction in response associated with weight, also statistically significant. Anticipating later analyses, response is often more naturally sensitive to dose per unit weight; the latter analysis is consistent with this. The prediction intervals as shown are effectively descriptive. Most of the data lie within these. The Confidence Intervals qualify statements about the mean response (over very many patients with specified dose or weight. One interpretation is that regression lines that are statistically consistent with the data must lie within the CI band. Page 9 of 13

10 c) The first analysis suggests that weight, dose and gender are all important predictors of response. The second suggests that, when dose/wt is included as a predictor, neither dose nor weight contribute much Dose extra information. We already know that weight and gender and interrelated. And by construction Dose/wt Weight does/wt is correlated to dose and Gender to weight. The apparent confusion arises frequently when the x-variable (predictor variables) are themselves inter-related. The slope coefficients are in such cases - not simply interpretable in terms of the corresponding bivariate correlations. Resp The diagram not a requirement, but worthy of marks if offered provides one way to envision a possible set of direct and indirect relationships with response. The concept of direct and indirect relationships has been discussed in class. d) The new derived variable simplifies the direct consideration and comparison of two simple models: Resp vs dose/wt separately for M/F. Separately these may be written as resp = int cpt + slope (dose/wt), with potentially different values for each for M/F. This can also be a way to investigate interaction. Here F dose/wt M ( ) + ( ) dose/wt Lines, when sketched, are effectively parallel and have almost same slope). Since the values and are small compared to SE, via T = 0.58, T = 0.00 we can conclude that a single regression relationship, for both M and F, is likely to be adequate. This in turn suggest that the Weight and Gender terms in the second model in c) may not be simply interpreted as suggested there as individually necessary. For the Gender term is correlated with Weight. Perhaps the inclusion of Weight requires the inclusion of Gender to counter-balance it. e) The analysis confirms that the important variable is dose/wt. No other variables are significant. However, the very high VIF values for dose and dose/wt point to the fact that these variables are (naturally) highly interdependent. One of these is likely to be to most important. The choice should be guided by the considerations of the way the drug interacts, biochemically, with the patient. Page 10 of 13

11 Q3 a) It is clear that the bivariate relationships (top row) are not linear. Additionally, there is clear evidence of variance of weight increasing with weight. It is also the case that there is a great deal of correlation between the 3 x variables, likely to cause problems if they are ever used together. b) The model in the log scale shows that all vars are very significantly different from 0; tho that was never in doubt. R 2 is high. This can also be written as Wt = Len 1.65 Ht 0.81 Width error. The nominal interpretation is that an increase of 1 in eg loglen ( ie an increase of 10- in Len) will induce - on average an increase of 1.65 in logwt (ie =45-fold in Wt) if all other variables are held constant. But The VIF values suggest that the covariates are correlated- as anticipated and that the SEs are therefore inflated. This was apparent also in the scatterplots. Effectively this means that some fish are large in respect of all three dimensions, and some are small. In these circumstances one option is to choose a single composite that reflects all of the variables. It is likely to be futile to choose one of them. There are however a number of unusual observations. 4 of these exhibit large residuals, which merit attention. Three are very large and positive, and one is large and negative. Two are influential, being far from the others in respect of (log)len, Ht, Width. There is nothing wrong with this, necessarily. c) The option followed was to choose a product named Vol. The Fitted Line plot has fitted Weight to Vol, in the log scale, presenting the analysis in the back-transformed anti-log scale. Alternatively as Log(Vol) = Log(Length)+ Log(Height) + Log(Width), the composite Log(Vol) variable is the sum of the three Log(covariates). The fitted model can be written as Wt = Vol error = 0.3 Vol error = 0.3 Len 0.98 Ht 0.98 Width error. An increase of 1 in LogVol (ie a 10-fold increase) will generate, on average a fold (ie 9.55-fold) increase in Wt, on average. The constant 0.3 could be thought of as fish-density, were fish to be cuboid. As it is, it is a combination of fish density and the ratio of actual fish volume to the volume of the corresponding cuboid. When Vol 0.98 is large (ie when Wt is large) the absolute errors implicit in a 10 error -fold variation are large.this exhibits the fan-like figure for the prediction intervals Predicting the weight if a large fish is harder than that of a small fish in absolute terms. The issue is equivalent to describing the prediction error in %age terms. Page 11 of 13

12 The R2 value is almost as high as in b). The value of s is This is in the logscale and compares with the value of above. The model is not quite as tight-fitting, but it is simpler. No info is available on unusual obs, or SEs. d) The two prediction equations are: (as in b) Pred logwt = Log(len) log(ht)m Log(Width) 2(0.037) (as in c) Pred logwt = Log(len*Ht*Width) 2(0.040), back transformed as antilog( Pred log(wt)) as below Fish dimensions Fish len ht width Vol log 10 dimensions Log(vol) sum log model b coeffs model c coeff const len ht width s const vol s Fish Pred LogWt lo hi Pred LogWt lo hi backtransform backtransform Conclusions: very similar See f) below Model c has a slope is ( 0.011). This includes coeff=1. That is, the data are statistically consistent with 1; a Null Hyp: slope =1 would not be rejected. A simpler version of this model would then be LogWt = const + Log Vol. Equiv this is Wt=0.3 Vol. This model is not unlike the Tree model discussed in class. Note that the SE (0.011) is very much smaller than the SE s for each of the dimensions in model b). That s because their SE s have been inflated by, effectively, the lack of determinancy of the separate coeffs. Note however, that the correlation between these dimensions has no implications at all for the usefulness of the prediction equation generated by model b). It is simply the case that many different combinations of these coefficients are effectively equivalent to each other. e) However, model (c) corresponds to giving coefficients of 1 to each (log) dimension. This is just about consistent with the fit for LogHt ( (0.21) it is not as consistent with LogLen (1.65 2(0.22) and LogWidth (0.55 2(0.18). The implications are that Fish that are very long will be given low values of log Wt in model c and Fish that are very wide will be given high values of Page 12 of 13

13 LogWt in model c. Fish are not cuboids. However it is moot whether the data would require a rejection (in model b) of the Null Hyp that all coefficients were equal to 1. XST7002 Page 13 of 13