Chapter 6 Bivariate Correlation & Regression

Transcription

1 Chapter 6 Bvarate Correlaton & Regresson 6.1 Scatterplots and Regresson Lnes 6. Estmatng a Lnear Regresson Equaton 6.3 R-Square and Correlaton 6.4 Sgnfcance Tests for Regresson Parameters

2 Scatterplot: a postve relaton Vsually dsplay relaton of two varables on X-Y coordnates 50 U.S. States Y = per capta ncome X = % adults wth BA degree CT Postve relaton: ncreasng X related to hgher values of Y MS

3 Scatterplot: a negatve relaton Y = % n poverty X = % females n labor force NM AR WI

4 Summarze scatter by regresson lne Use lnear regresson to estmate best-ft lne thru ponts: How can we use sample data on the Y & X varables to estmate populaton parameters for the best-fttng lne?

5 Slopes and ntercepts We learned n algebra that a lne s unquely located n a coordnate system by specfyng: (1) ts slope ( rse over run ); and () ts ntercept (where t crosses the Y-axs) Equaton has a bvarate lnear relatonshp: Y = a + bx where: b s slope a s ntercept DRAW THESE LINES: Y = 0 + X Y = X

6 Predcton equaton vs. regresson model In predcton equaton, caret over Y ndcates predcted ( expected ) score of th case for ndependent value X : Yˆ a b X But we can never perfectly predct socal relatonshps! Regresson model s error term ndcates how dscrepant s the predcted score from observed value of the th case: Y a b X e Calculate the magntude and sgn of the th case s error by subtractng 1 st equaton from nd equaton (see next slde): Y Y ˆ e

7 Regresson error The regresson error, or resdual, for the th case s the dfference between the value of the dependent varable predcted by a regresson equaton and the observed value of that case. Subtract the predcton equaton from the lnear regresson model to dentfy the th case s error term Y a b X e Ŷ a b X Y Ŷ e An analogy: In weather forecastng, an error s the dfference between the weatherperson s predcted hgh temperature for today and the actual hgh temperature observed today: Observed temp 86º - Predcted temp 91º = Error -5º

8 The Least Squares crteron Scatterplot for state Income & Educaton has a postve slope To plot the regresson lne, we apply a crteron yeldng the best ft of a lne through the cloud of ponts Ordnary least squares (OLS) a method for estmatng regresson equaton coeffcents -- ntercept (a) and slope (b) -- that mnmze the sum of squared errors

9 OLS estmator of the slope, b Because the sum of errors s always 0, we want parameter estmators that wll mnmze the sum of squared errors: N 1 ( Y Yˆ ) e Fortunately, both OLS estmators have ths desred property Bvarate regresson coeffcent: b ( Y Y)( X ( X X) X) Numerator s sum of product of devatons around means; when dvded by N 1 t s called the covarance of Y and X. If we also dvde the denomnator by N 1, the result s the nowfamlar varance of X. Thus, b s X s

10 OLS estmator of the ntercept, a The OLS estmator for the ntercept (a) smply changes the mean of Y (the dependent varable) by an amount equalng the regresson slope s effect for the mean of X: Two mportant facts arse from ths relaton: (1) The regresson lne always goes through the pont of both varables means! a Y bx () When the regresson slope s zero, for every X we only predct that Y equals the ntercept a, whch s also the mean of the dependent varable! a Y b X 0

11 Use these two bvarate regresson equatons, estmated from the 50 States data, to calculate some predcted values: 1. Regress ncome on bachelor s degree: Yˆ $ X Yˆ a b What predcted ncomes for: X = 1%: Y= X = 8%: Y= X. Regress poverty percent on female labor force pct: Yˆ 45.% X What predcted poverty % for: X = 55%: Y= X = 70%: Y=

12 Use these two bvarate regresson equatons, estmated from the 008 GSS data, to calculate some predcted values: 3. Regress church attendance per year on age (N=,005) Yˆ What predcted attendance for: X X = 18 years: Y= X = 89 years: Y= 4. Regress sex frequency per year on age (N=1,680) Yˆ X What predcted actvty for: X = 18 years: Y= X = 89 years: Y= Lnearty s not always a reasonable, realstc assumpton to make about socal behavors! Yˆ a b X

13 Errors n regresson predcton Every regresson lne through a scatterplot also passes through the means of both varables;.e., pont ( Y, X) We can use ths relatonshp to dvde the varance of Y nto a double devaton from: (1) the regresson lne () the Y-mean lne Then calculate a sum of squares that reveals how strongly Y s predcted by X.

14 Illnos double devaton In Income-Educaton scatterplot, show the dfference between the mean and Illnos Y-score as the sum of two devatons: IL } } Y Error devaton of observed and predcted scores Y ˆ Y Regresson devaton of predcted score from the mean Y ˆ Y

15 Parttonng the sum of squares Now generalze ths procedure to all N observatons 1. Subtract the mean of Y from the th observed score (= case s devaton score): Y Y. Smultaneously subtract and add th predcted score (leaves the devaton unchanged): Y Yˆ Yˆ Y 3. Group these four elements nto two terms: ( Y Yˆ ) ( Yˆ Y ) 4. Square both grouped terms: 5. Sum the squares across all N cases: ( Y Y ( Y Y ( Y Yˆ ) ( Yˆ Y ) ˆ ) ˆ ) 6. Step #5 equals the sum of the squared devatons n step #1 (whch s also the ( Y Y ) numerator of the varance of Y): Therefore: ( Y Y ) ( Y Yˆ ) ( Yˆ Y )

16 Namng the sums of squares Each result of the precedng partton has a name: ( Y Y) ( Y Yˆ ) ( Yˆ Y) TOTAL sum of squares ERROR sum of squares REGRESSION sum of squares SSTOTAL = SSERROR + SSREGRESSION The relatve proportons of the two terms on the rght ndcate how well or poorly we can predct the varance n Y from ts lnear relatonshp wth X The SS TOTAL should be famlar to you t s the numerator of the varance of Y (see the Notes for Chapter ). When we partton the sum of squares nto the two components, we re analyzng the varance of the dependent varable n a regresson equaton. Hence, ths method s called the analyss of varance or ANOVA.

17 Coeffcent of Determnaton If we had no knowledge about the regresson slope (.e., b = 0 and thus SS REGRESSION = 0), then our only predcton s that the score of Y for every case equals the mean (whch also equals the equaton s ntercept a; see slde #10 above). Yˆ Yˆ Yˆ a a a b 0X X But, f b 0, then we can use nformaton about the th case s score on X to mprove our predcted Y for case. We ll stll make errors, but the stronger the Y-X lnear relatonshp, the more accurate our predctons wll be.

18 R as a PRE measure of predcton Use nformaton from the sums of squares to construct a standardzed proportonal reducton n error (PRE) measure of predcton success for a regresson equaton Ths PRE statstc, the coeffcent of determnaton, s the proporton of the varance n Y explaned statstcally by Y s lnear relatonshp wth X. R SS TOTAL SS SS TOTAL ERROR SS REGRESSION SS TOTAL R ( Y ˆ ˆ Y) ( Y Y ) ( Y Y) ( Y Y) ( Y Y) The range of R-square s from 0.00 to 1.00, that s, from no predctablty to perfect predcton.

19 Fnd the R for these 50-States bvarate regresson equatons 1. R-square for regresson of ncome on educaton SS REGRESSION = SS ERROR = 34. SS TOTAL = R =. R-square for poverty-female labor force equaton SS REGRESSION = SS ERROR = 31.6 SS TOTAL = R =

20 Here are some R problems from the 008 GSS 3. R-square for church attendance regressed on age SS REGRESSION = 67,13 SS ERROR =,861,98 SS TOTAL = R = 4. R-square for sex frequency-age equaton SS REGRESSION = 1,511,6 SS ERROR = SS TOTAL = 10,50,53 R =

21 The correlaton coeffcent, r Correlaton coeffcent s a measure of the drecton and strength of the lnear relatonshp of two varables Attach the sgn of regresson slope to square root of R : r r XY R Or, n terms of covarances and standard devatons: r s s Y s X s s X XY s Y r XY

22 Calculate the correlaton coeffcents for these pars: Regresson Eqs. R b r Income-Educaton Poverty-labor force Church attend-age Sex frequency-age

23 Comparson of r and R Ths table summarzes dfferences between the correlaton coeffcent and coeffcent of determnaton for two varables. Correlaton Coeffcent Coeffcent of Determnaton Sample statstc r R Populaton parameter ρ ρ Relatonshp r = R R = r Test statstc t test F test

24 Sample and populaton Regresson equatons estmated wth sample data can be used to test hypotheses about each of the three correspondng populaton parameters Sample equaton: Ŷ a b X R Populaton equaton: Ŷ X Each par of null and alternatve (research) hypotheses are statements about a populaton parameter. Performng a sgnfcance test requres usng sample statstcs to estmate a standard error or a par of mean squares.

25 Hypotheses about slope, A typcal null hypothess about the populaton regresson slope s that the ndependent varable (X) has no lnear relaton wth the dependent varable (Y). Its pared research hypothess s nondrectonal (a two-taled test): H0 : β 0 H1 : β 0 Other hypothess pars are drectonal (one-taled tests): H H 0 1 : : β β 0 0 or H H 0 1 : : β β 0 0

26 Samplng Dstrbuton of The Central Lmt Theorem, whch let us analyze the samplng dstrbuton of large-sample means as a normal curve, also treats the samplng dstrbuton of as normal, wth mean = 0 and standard error σ β. Hypothess tests may be one- or two-taled. β = 0

27 The t-test for To test whether a large sample s regresson slope (b ) has a low probablty of beng drawn from a samplng dstrbuton wth a hypotheszed populaton parameter of zero ( = 0), apply a t-test (same as Z-test for large N). t b β s b where s b s the sample estmate of the standard error of the regresson slope. SSDA#4 (pp. 19) shows how to calculate ths estmate wth sample data. But, n ths course we wll rely on SPSS to estmate the standard error.

28 Here s a research hypothess: The greater the percentage of college degrees, the hgher a state s per capta ncome. 1. Estmate the regresson equaton (s b n parens): Ŷ $ X (.1) (0.10). Calculate the test statstc: t b β s b 3. Decde about the null hypothess (one-taled test): 1-tal -tal Probablty of Type I error: 5. Concluson:

29 For ths research hypothess, use the 008 GSS (N=1,919): The more sblngs respondents have, the lower ther occupatonal prestge scores. 1. Estmate the regresson equaton (s b n parentheses):. Calculate the test statstc: b β t sb 3. Decde about the null Ŷ X (0.47) (0.10) hypothess (one-taled test): 4. Probablty of Type I error: 5. Concluson:

30 Research hypothess: The number of hours people work per week s unrelated to number of sblngs they have. 1. Estmate the regresson equaton (s b n parentheses): Ŷ (0.65) 0.08 X (0.14). Calculate the test statstc: t b β s b 3. Decde about the null hypothess (two-taled test): 4. Probablty of Type I error: 5. Concluson:

31 Hypothess about the ntercept, Researchers rarely have any hypothess about the populaton ntercept (the dependent varable s predcted score when the ndependent varable = 0). Use SPSS s standard error for a t-test of ths hypothess par: H H 0 1 : : 0 0 t a α Test ths null hypothess: the ntercept n the state ncomeeducaton regresson equaton s zero. t a α s a Decson about H 0 (two-taled): Probablty of Type I error: Concluson: s a

32 Chapter The Ch-Square and F Dstrbutons

33 Ch-Square Two useful famles of theoretcal statstcal dstrbutons, both based on the Normal dstrbuton: Ch-square and F dstrbutons The Ch-square ( ) famly: for normally dstrbuted random varables, square and add each Z-score (Greek nu) s the degrees of freedom (df) for a specfc famly member ( Y1 For = : Z1 ) ( Y Y Z Y Y Z Z 1 Y )

34 Shapes of Ch-Square Mean for each = and varance =. Wth larger df, plots show ncreasng symmetry but each s postvely skewed: Areas under a curve can be treated as probabltes

35 The F Dstrbuton The F dstrbuton famly: formed as the rato of two ndependent ch-square random varables. Ronald Fscher, a Brtsh statstcan, frst descrbed the dstrbuton 19. In 1934, George Snedecor tabulated the famly s values and called t the F dstrbuton n honor of Fscher. Every member of the F famly has two degrees of freedom, one for the chsquare n the numerator and one for the ch-square n the denomnator: F 1 / / 1 F s used to test hypotheses about whether the varances of two or more populatons are equal (analyss of varance = ANOVA) F s also used n tests of explaned varance n multple regresson equatons (also called ANOVA)

36 Each member of the F dstrbuton famly takes a dfferent shape, varyng wth the numerator and denomnator dfs:

37 Chapter 6 Return to hypothess testng for regresson

38 Hypothess about A null hypothess about the populaton coeffcent of determnaton (Rho-square) s that none of the dependent varable (Y) varaton s due to ts lnear relaton wth the ndependent varable (X): The only research hypothess s that Rho-square n the populaton s H0 : ρ 1 H : ρ 0 greater than zero: Why s H 1 never wrtten wth a negatve Rho-square (.e., < 0)? 0 To test the null hypothess about, use the F dstrbuton, a rato of two ch-squares each dvded by ther degrees of freedom: Degree of freedom: the number of values free to vary when computng a statstc

39 Calculatng degrees of freedom The concept of degrees of freedom (df) s probably better understood by an example than by a defnton. Suppose a sample of N =4 cases has a mean of 6. I tell you that Y 1 = 8 and Y = 5; what are Y 3 and Y 4? Those two scores can take many values that would yeld a mean of 6 (Y 3 = 5 & Y 4 = 6; or Y 3 = 9 & Y 4 = ) But, f I now tell you that Y 3 = 4, what must Y 4 = Once the mean and N-1 other scores are fxed, the Nth score has no freedom to vary. The three sums of squares n regresson analyss cost dfferng degrees of freedom, whch must be pad when testng a hypothess about.

40 d for the 3 Sums of Squares 1. SS TOTAL has df = N - 1, because for a fxed total all scores except the fnal score are free to vary. Because the SS REGRESSION s estmated from one regresson slope (b ), t costs 1 df 3. Calculate the df for SS ERROR as the dfference: df TOTAL = df REGRESSION + df ERROR N - 1 = 1 + df ERROR Therefore: df ERROR = N-

41 Mean Squares To standardze F for dfferent sze samples, calculate mean (average) sums of squares per degree of freedom, for the three components SS df TOTAL TOTAL SS SS SS REGRESSION ERROR df REGRESSION df TOTAL REGRESSION ERROR ERROR N 1 1 N SS SS Label the two terms on the rght sde as Mean Squares: SS REGRESSION /1 = MS REGRESSION SS ERROR /(N-) = MS ERROR The F statstc s thus a rato of the two Mean Squares: F MS REGRESSION MS ERROR SS TOTAL / df TOTAL = the varance of Y (see the Notes for Chapter ), further ndcaton we re conductng an analyss of varance (ANOVA).

42 Analyss of Varance Table One more tme: The F test for 50 State Income-Educaton Calculate and fll n the two MS n ths summary ANOVA table, and then compute the F-rato: Source SS df MS F Regresson Error 34. Total A decson about H 0 requres the crtcal values for F, whose dstrbutons nvolve the two degrees of freedom assocated wth the two Mean Squares

43 Crtcal values for F In a populaton, f s greater than zero (the H 1 ), then the MS REGRESSION wll be sgnfcantly larger than MS ERROR, as revealed by the F test statstc. An F statstc to test a null hypothess s a rato two Mean Squares. Each MSs has a dfferent degrees of freedom (df = 1 n the numerator, df = N- n the denomnator). For large samples, use ths table of crtcal values for the three conventonal alpha levels: Why are the c.v. for F always postve? df R, df E c.v..05 1, , , 10.83

44 Test the hypothess about for the occupatonal prestge-sblngs regresson, where sample R = H0 : ρ 0 H1 : ρ 0 Source SS df MS F Regresson 14,0 Error 355,775 Total 369, Decde about null hypothess: Probablty of Type I error: Concluson:

45 Test the hypothess about for the hours workedsblngs regresson, where sample R = Source SS df MS F Regresson 68 Error 51,68 Total 51, Decde about null hypothess: Probablty of Type I error: Concluson: Wll you make always the same or dfferent decsons f you test hypotheses about both and for the same bvarate regresson equaton? Why or why not?

46