Simple linear regression

Transcription

1 Simple liear regressio Tro Aders Moger 3..7 Example 6: Populatio proportios Oe sample X Assume X ~ Bi(, P, so that P ˆ is a frequecy. P The ~ N(, P( P / (approximately, for large P Thus ~ N(, ( / (approximately, for large ( Thus P Z / < P Cofidece iterval for P Z < ˆ + ( α P Zα / (, + Z α / α / ( α Example 6 (Hypothesis testig Example 7: Differeces betwee populatio proportios-two samples Hypotheses: H :PP H :P P Test statistic P ~ N(, P ( P uder H, for large Reject H if P < P ( P Z α /,or if P P ( P > Z α / Assume X ~ Bi(, P ad X ~ Bi(, P, so that ˆ X P ad ˆ X P are frequecies ˆ P ( P P ~ N (, P ( P P ( P The + Cofidece iterval for P -P ± Z α / ( ( + (approximately

2 Example 7 (Hypothesis testig Hypotheses: H :P P H :P P ~ Test statistic ( ˆ ˆ ( ˆ P P P + where ˆ ˆ ˆ P + P P + Reject H if N > ˆ ˆ ˆ ( P P ( P + Z α / (, Spotaous abortios amog surgical urses ad other urses Wat to test if there is differece betwee the proportios of abortios i the two groups H : P op.urses P others H : P op.urses P others No. iterviewed No. pregacies No. abortios Percet abortios Surgical urses Other urses Calculatio: P.78 P Total o. abortios + 3 p.86 Total o. pregacies z ( (.86.4 P-value.444.%, reject H o 5%- sig.level (ca t do this i SPSS 95% cofidece iterval for P -P : ˆ ˆ ˆ ( ˆ ˆ ˆ P ( P P P ( P P ±.96 * + (.5,.9 Repetitio: Testig: Idetify data; cotiuous->t-tests; proportios- >Normal approx. to biomial dist. If cotious: oe-sample, matched pairs, two idepedet samples? Assumptios: Are data ormally distributed? If two id. samples, equal variaces i both groups? FormulateH ad H (H is always o differece, o effect of treatmet etc., choose sig. level (α5% Calculate test statistic

3 Iferece: Test statistic usually stadardized; (estimator-expected value of estimator uder H /(estimated stadard error Gives you a locatio o the x-axis i a distributio Compare this value to the value at the.5%-percetile ad 97.5%-percetile of the distributio If smaller tha the.5%-percetile or larger tha the 97.5%-percetile, reject H P-value: Area i the tails of the distributio below value of test statistic+area above value of test-statistic (twosided testig If smaller tha.5, reject H If cofidece iterval for mea or mea differece (depeds o test what you use does ot iclude H value from, reject H Last week: Looked at cotiuous, ormally distributed variables Used t-tests to see if there was sigificat differece betwee meas i two groups How strog is the relatioship betwee two such variables? Correlatio What if oe wats to study the relatioship betwee several such variables? Liear regressio Coectio betwee variables kostad kostad år areal We would like to study coectio betwee x ad y! Data from the first obligatory assigmet: Birth weight ad smokig Childre of 89 wome Low birth weight is a medical risk factor Does mother s smokig status have ay ifluece o the birth weight? Also iterested i relatioship with other variables: Mother s age, mother s weight, high blood pressure, ethicity etc. 3

4 Is birth weight ormally distributed? Q-Q plot (check Normality plots with tests uder plots: Histogram From explore i SPSS Normal Q-Q Plot of birthweight 5 3 Frequecy 5 Expected Normal Mea 944,656 Std. Dev. 79,4 N 89,, 3, 4, 5, birthweight Observed Value Tests for ormality: Pearsos correlatio coefficiet r Tests of Normality birthweight Kolmogorov-Smirov(a Statistic,43 89 Sig.,(* * This is a lower boud of the true sigificace. a Liljefors Sigificace Correctio df Shapiro-Wilk Statistic The ull hypothesis is that the data are ormal. Large p- value idicates ormal distributio. For large samples, the p-value teds to be low. The graphical methods are more importat,99 df 89 Sig.,438 Measures the liear relatioship betwee variables r: All data lie o a icreasig straight lie r-: All data lie o a decreasig straight lie r: No liear relatioship I liear regressio, ofte use R (r as a measure of the explaatory power of the model R close to meas that the observatios are close to the lie, r close to meas that there is o liear relatioship betwee the observatios 4

5 Testig for correlatio It is also possible to test whether a sample correlatio r is large eough to idicate a ozero populatio correlatio Test statistic: r ~ t r Note: The test oly works for ormal distributios ad liear correlatios: Always also ivestigate scatter plot! Pearsos correlatio coefficiet i SPSS: Aalyze->Correlate->bivariate Check Pearso Tests if r is sigificatly differet from Null hypothesis is that r The variables have to be ormally distributed Idepedece betwee observatios Example: Correlatio from SPSS: 5, birthweight 4, 3,,, birthweight weight i pouds Correlatios Pearso Correlatio Sig. (-tailed N Pearso Correlatio Sig. (-tailed N weight i birthweight pouds,86*, 89 89,86*, *. Correlatio is sigificat at the.5 level (-tailed., 5,, 5,, 5, weight i pouds 5

6 If the data are ot ormally distributed: Spearmas rak correlatio, r s Spearma correlatio: Measures all mootoous relatioships, ot oly liear oes No distributio assumptios r s is betwee - ad, similar to Pearsos correlatio coefficiet I SPSS: Aalyze->Correlate->bivariate Check Spearma Also provides a test o whether r s is differet from Correlatios Spearma's rho birthweight Correlatio Coefficiet Sig. (-tailed N weight i pouds Correlatio Coefficiet Sig. (-tailed N **. Correlatio is sigificat at the. level (-tailed. weight i birthweight pouds,,48**., 89 89,48**,, Liear regressio Coectio betwee variables Wish to fit a lie as close to the observed data (two ormally distributed varaibles as possible Example: Birth weighta+b*mother s weight I SPSS: Aalyze->Regressio->Liear Click Statistics ad check Cofidece iterval for B Choose oe variable as depedet (Birth weight as depedet, ad oe variable (mother s weight as idepedet Importat to kow which variable is your depedet variable! kostad areal Fit a lie! 6

7 The stadard simple regressio model We defie a model Yi β + βxi + ε i where ε i are idepedet, ormally distributed, with equal variace σ We ca the use data to estimate the model parameters, ad to make statemets about their ucertaity What ca you do with a fitted lie? Iterpolatio Extrapolatio (sometimes dagerous! Iterpret the parameters of the lie How to defie the lie that fits best? How to compute the lie fit with the least squares method? The sum of the squares of the errors miimized Least squares method! Note: May other ways to fit the lie ca be imagied Let (x, y, (x, y,...,(x, y deote the poits i the plae. Fid a ad b so that ya+bx fit the poits by miimizig Solutio: y + ( a + bx y + + ( a + bx y ( a + bxi yi i S ( a + bx L b xi yi ( xi ( yi xi yi ( xi ( xi xi xi xy yi b xi a y bx where x x, i y yi ad all sums are doe for i,...,. 7

8 y How do you get this aswer? Differetiate S with respect to a og b, ad set the result to S ( a + bx i y i a i S ( a + bxi yi xi b We get: i a + b( x i yi ( xi + b( xi xi yi a This is two equatios with two ukows, ad the solutio of these give the aswer. y agaist x x agaist y Liear regressio of y agaist x does ot give the same result as the opposite. Regressio of y agaist x Regressio of x agaist y x Aaylzig the variace What is the logic behid R? Defie SSE: Error sum of squares ( a+ bxi yi SSR: Regressio sum of squares ( a+ bxi y SST: Total sum of squares ( yi y We ca show that SST SSR + SSE SSR SSE Defie R corr( x, y R SST SST is the coefficiet of determiatio y y a+ bx ˆi i SST yi y x x i ε SSE y yˆ i i i SSR yˆi y 8

9 Assumptios Usually check that the depedet variable is ormally distributed More formally, the residuals, i.e. the distace from each observatio to the lie, should be ormally distributed I SPSS: I liear regressio, click Statistics. Uder residuals check casewise diagostics, ad you will get outliers larger tha 3 or less tha -3 i a separate table. I liear regressio, also click Plots. Uder stadardized residuals plots, check Histogram ad Normal probability plot. Choose *Zresid as y-variable ad *Zpred as x-variable Example: Regressio of birth weight with mother s weight as idepedet variable Model Model Model Summary b Adjusted Std. Error of R R Square R Square the Estimate,86 a,35,9 78,47 a. Predictors: (Costat, weight i pouds b. Depedet Variable: birthweight Model Regressio Residual Total ANOVA b Sum of Squares df Mea Square F Sig ,3 6,686, a , a. Predictors: (Costat, weight i pouds b. Depedet Variable: birthweight (Costat weight i pouds a. Depedet Variable: birthweight Coefficiets a Ustadardized Stadardized Coefficiets Coefficiets 95% Cofidece Iterval for B B Std. Error Beta t Sig. Lower Boud Upper Boud 369,67 8,43,374, 99,4 8,34 4,49,73,86,586,,5 7,89 Residuals: Check of assumptios: Casewise Diagostics a Histogram Predicted Case Number Std. Residual birthweight Value Residual -3,5 79, 9,837-9,8 a. Depedet Variable: birthweight 3 Depedet Variable: birthweight Residuals Statistics a 5 Miimum Maximum Mea Std. Deviatio N Predicted Value 74,3 3476, ,656 35, Residual -9,8 75,59, 76, Std. Predicted Value -,69 3,93,, 89 Std. Residual -3,5,89,, a. Depedet Variable: birthweight Frequecy Regressio Stadardized Residual Mea 6,77E-7 Std. Dev.,997 N 89 9

10 Check of assumptios cot d: Check of assumptios cot d: Normal P-P Plot of Regressio Stadardized Residual Scatterplot, Depedet Variable: birthweight 3 Depedet Variable: birthweight Expected Cum Prob,8,6,4, Regressio Stadardized Residual ,,,,4,6,8, Observed Cum Prob Regressio Stadardized Predicted Value Iterpretatio: Have fitted the lie Birth weight *mother s weight If mother s weight icreases by pouds, what is the predicted impact o ifat s birth weight? 4.49*89 grams What s the predicted birth weight of a ifat with a 5 poud mother? *5334 grams Ifluece of extreme observatios NOTE: The result of a regressio aalysis is very much iflueced by poits with extreme values, i either the x or the y directio. Always ivestigate visually, ad determie if outliers are actually erroeous observatios

11 But how to aswer questios like: Give that a positive slope (b has bee estimated: Does it give a reproducible idicatio that there is a positive tred, or is it a result of radom variatio? What is a cofidece iterval for the estimated slope? What is the predictio, with ucertaity, at a ew x value? Cofidece itervals for simple regressio I a simple regressio model, β a estimates b estimates β ˆ σ SSE /( Also, ( b β / Sb ~ t ˆ σ where Sb ( sx of b estimates So a cofidece iterval for by b± t, α /Sb σ estimates variace β is give Hypothesis testig for simple regressio Choose hypotheses: H : β H: β Test statistic: b/ S ~ b t RejectH if b S t α or / b <, / b/ Sb > t, α /