CHAPTER 1 2 CORRELATION AND REGRESSION

Transcription

1 CHAPTER 1 CORRELATION AND REGRESSION

2 CORRELATION AND REGRESSION LEARNING OBJECTIVES After readng ths chapter a student wll be able to understand The meanng of bvarate data and technques of preparaton of bvarate dstrbuton; The concept of correlaton between two varables and quanttatve measurement of correlaton ncludng the nterpretaton of postve, negatve and zero correlaton; Concept of regresson and ts applcaton n estmaton of a varable from known set of data. 1.1 INTRODUCTION In the prevous chapter, we dscussed many a statstcal measure relatng to Unvarate dstrbuton.e. dstrbuton of one varable lke heght, weght, mark, proft, wage and so on. However, there are stuatons that demand study of more than one varable smultaneously. A busnessman may be keen to know what amount of nvestment would yeld a desred level of proft or a student may want to know whether performng better n the selecton test would enhance hs or her chance of dong well n the fnal examnaton. Wth a vew to answerng ths seres of questons, we need to study more than one varable at the same tme. Correlaton Analyss and Regresson Analyss are the two analyses that are made from a multvarate dstrbuton.e. a dstrbuton of more than one varable. In partcular when there are two varables, say x and y, we study bvarate dstrbuton. We restrct our dscusson to bvarate dstrbuton only. Correlaton analyss, t may be noted, helps us to fnd an assocaton or the lack of t between the two varables x and y. Thus f x and y stand for proft and nvestment of a frm or the marks n Statstcs and Mathematcs for a group of students, then we may be nterested to know whether x and y are assocated or ndependent of each other. The extent or amount of correlaton between x and y s provded by dfferent measures of Correlaton namely Product Moment Correlaton Coeffcent or Rank Correlaton Coeffcent or Coeffcent of Concurrent Devatons. In Correlaton analyss, we must be careful about a cause and effect relaton between the varables under consderaton because there may be stuatons where x and y are related due to the nfluence of a thrd varable although no causal relatonshp exsts between the two varables. Regresson analyss, on the other hand, s concerned wth predctng the value of the dependent varable correspondng to a known value of the ndependent varable on the assumpton of a mathematcal relatonshp between the two varables and also an average relatonshp between them. 1. BIVARIATE DATA When data are collected on two varables smultaneously, they are known as bvarate data and the correspondng frequency dstrbuton, derved from t, s known as Bvarate Frequency Dstrbuton. If x and y denote marks n Maths and Stats for a group of 30 students, then the correspondng bvarate data would be (x, y ) for 1,,. 30 where (x 1, y 1 ) denotes the marks n Mathematcs and Statstcs for the student wth seral number or Roll Number 1, (x, y ), that for the student wth Roll Number and so on and lastly (x 30, y 30 ) denotes the par of marks for the student bearng Roll Number COM M ON PROFICIENCY TEST

3 As n the case of a Unvarate Dstrbuton, we need to construct the frequency dstrbuton for bvarate data. Such a dstrbuton takes nto account the classfcaton n respect of both the varables smultaneously. Usually, we make horzontal classfcaton n respect of x and vertcal classfcaton n respect of the other varable y. Such a dstrbuton s known as Bvarate Frequency Dstrbuton or Jont Frequency Dstrbuton or Two way classfcaton of the two varables x and y. Illustraton Example 1.1 Prepare a Bvarate Frequency table for the followng data relatng to the marks n Statstcs (x) and Mathematcs (y): (15, 13), (1, 3), (, 6), (8, 3), (15, 10), (3, 9), (13, 19), (10, 11), (6, 4), (18, 14), (10, 19), (1, 8), (11, 14), (13, 16), (17, 15), (18, 18), (11, 7), (10, 14), (14, 16), (16, 15), (7, 11), (5, 1), (11, 15), (9, 4), (10, 15), (13, 1) (14, 17), (10, 11), (6, 9), (13, 17), (16, 15), (6, 4), (4, 8), (8, 11), (9, 1), (14, 11), (16, 15), (9, 10), (4, 6), (5, 7), (3, 11), (4, 16), (5, 8), (6, 9), (7, 1), (15, 6), (18, 11), (18, 19), (17, 16) (10, 14), Take mutually exclusve classfcaton for both the varables, the frst class nterval beng 0-4 for both. Soluton From the gven data, we fnd that Range for x Range for y We take the class ntervals 0-4, 4-8, 8-1, 1-16, 16-0 for both the varables. Snce the frst par of marks s (15, 13) and 15 belongs to the fourth class nterval (1-16) for x and 13 belongs to the fourth class nterval for y, we put a stroke n the (4, 4)-th cell. We carry on gvng tally marks tll the lst s exhausted. STATISTICS 1.3

4 CORRELATION AND REGRESSION Table 1.1 Bvarate Frequency Dstrbuton of Marks n Statstcs and Mathematcs. X MARKS IN STATS MARKS IN MATHS Y Total 0 4 I (1) I (1) II () I (1) IIII (4) IIII (5) I (1) I (1) I (1) II () IIII (4) IIII I (6) I (1) I (1) III (3) II () IIII (5) I (1) IIII (5) III (3) 9 Total We note, from the above table, that some of the cell frequences (f j ) are zero. Startng from the above Bvarate Frequency Dstrbuton, we can obtan two types of unvarate dstrbutons whch are known as: (a) (b) Margnal dstrbuton. Condtonal dstrbuton. If we consder the dstrbuton of Statstcs marks along wth the margnal totals presented n the last column of Table 1-1, we get the margnal dstrbuton of marks n Statstcs. Smlarly, we can obtan one more margnal dstrbuton of Mathematcs marks. The followng table shows the margnal dstrbuton of marks of Statstcs. Table 1. Margnal Dstrbuton of Marks n Statstcs Marks No. of Students Total 50 We can fnd the mean and standard devaton of marks n Statstcs from Table 1.. They would be known as margnal mean and margnal SD of Statstcs marks. Smlarly, we can obtan the margnal mean and margnal SD of Mathematcs marks. Any other statstcal measure n respect of x or y can be computed n a smlar manner. 1.4 COM M ON PROFICIENCY TEST

5 If we want to study the dstrbuton of Statstcs Marks for a partcular group of students, say for those students who got marks between 8 to 1 n Mathematcs, we come across another unvarate dstrbuton known as condtonal dstrbuton. Table 1.3 Condtonal Dstrbuton of Marks n Statstcs for Students havng Mathematcs Marks between 8 to 1 Marks No. of Students Total 15 We may obtan the mean and SD from the above table. They would be known as condtonal mean and condtonal SD of marks of Statstcs. The same result holds for marks n Mathematcs. In partcular, f there are m classfcatons for x and n classfcatons for y, then there would be altogether (m + n) condtonal dstrbuton. 1.3 CORRELATION ANALYSIS Whle studyng two varables at the same tme, f t s found that the change n one varable s recprocated by a correspondng change n the other varable ether drectly or nversely, then the two varables are known to be assocated or correlated. Otherwse, the two varables are known to be dssocated or uncorrelated or ndependent. There are two types of correlaton. () () Postve correlaton Negatve correlaton If two varables move n the same drecton.e. an ncrease (or decrease) on the part of one varable ntroduces an ncrease (or decrease) on the part of the other varable, then the two varables are known to be postvely correlated. As for example, heght and weght yeld and ranfall, proft and nvestment etc. are postvely correlated. On the other hand, f the two varables move n the opposte drectons.e. an ncrease (or a decrease) on the part of one varable results a decrease (or an ncrease) on the part of the other varable, then the two varables are known to have a negatve correlaton. The prce and demand of an tem, the profts of Insurance Company and the number of clams t has to meet etc. are examples of varables havng a negatve correlaton. The two varables are known to be uncorrelated f the movement on the part of one varable does not produce any movement of the other varable n a partcular drecton. As for example, Shoe-sze and ntellgence are uncorrelated. STATISTICS 1.5

6 CORRELATION AND REGRESSION 1.4 MEASURES OF CORRELATION We consder the followng measures of correlaton: (a) (b) (c) (d) (a) Scatter dagram Karl Pearson s Product moment correlaton coeffcent Spearman s rank correlaton co-effcent Co-effcent of concurrent devatons SCATTER DIAGRAM Ths s a smple dagrammatc method to establsh correlaton between a par of varables. Unlke product moment correlaton co-effcent, whch can measure correlaton only when the varables are havng a lnear relatonshp, scatter dagram can be appled for any type of correlaton lnear as well as non-lnear.e. curvlnear. Scatter dagram can dstngush between dfferent types of correlaton although t fals to measure the extent of relatonshp between the varables. Each data pont, whch n ths case a par of values (x, y ) s represented by a pont n the rectangular axes of cordnates. The totalty of all the plotted ponts forms the scatter dagram. The pattern of the plotted ponts reveals the nature of correlaton. In case of a postve correlaton, the plotted ponts le from lower left corner to upper rght corner, n case of a negatve correlaton the plotted ponts concentrate from upper left to lower rght and n case of zero correlaton, the plotted ponts would be equally dstrbuted wthout depctng any partcular pattern. The followng fgures show dfferent types of correlaton and the one to one correspondence between scatter dagram and product moment correlaton coeffcent. Y Y O X O X FIGURE 1.1 FIGURE 1. Showng Postve Correlaton Showng perfect Correlaton (0 < r <1) (r 1) 1.6 COM M ON PROFICIENCY TEST

7 Y Y O X O X FIGURE 1.3 FIGURE 1.4 Showng Negatve Correlaton Showng perfect Negatve Correlaton ( 1 < r <0) (r 1) Y Y O (b) X O X FIGURE 1.5 FIGURE 1.6 Showng No Correlaton Showng Curvlnear Correlaton (r 0) (r 0) KARL PEARSON S PRODUCT MOMENT CORRELATION COEFFICIENT Ths s by for the best method for fndng correlaton between two varables provded the relatonshp between the two varables s lnear. Pearson s correlaton coeffcent may be defned as the rato of covarance between the two varables to the product of the standard devatons of the two varables. If the two varables are denoted by x and y and f the correspondng bvarate data are (x, y ) for 1,, 3,.., n, then the coeffcent of correlaton between x and y, due to Karl Pearson, n gven by : STATISTICS 1.7

8 CORRELATION AND REGRESSION r where r xy Cov x, y S x S y...(1.1) cov (x, y) x x (y y) xy x y...(1.) n n x x x S x x n n...(1.3) y y y S y y...(1.4) n n and A sngle formula for computng correlaton coeffcent s gven by r nx y x y nx x n y ( y )...(1.5) In case of a bvarate frequency dstrbuton, we have Cov(x,y) x y f,j N j x y... (1.6) S x f x o N x...(1.7) and fojy j S y N y j...(1.8) where x Md-value of the th class nterval of x. 1.8 COM M ON PROFICIENCY TEST

9 y j f o f oj f j Md-value of the j th class nterval of y Margnal frequency of x Margnal frequency of y frequency of the (, j) th cell N j fo oj Total frequency... (1.9),j f PROPERTIES OF CORRELATION COEFFICIENT () () The Coeffcent of Correlaton s a unt-free measure. j f Ths means that f x denotes heght of a group of students expressed n cm and y denotes ther weght expressed n kg, then the correlaton coeffcent between heght and weght would be free from any unt. The coeffcent of correlaton remans nvarant under a change of orgn and/or scale of the varables under consderaton dependng on the sgn of scale factors. Ths property states that f the orgnal par of varables x and y s changed to a new par of varables u and v by effectng a change of orgn and scale for both x and y.e. x a y c u and v b d where a and c are the orgns of x and y and b and d are the respectve scales and then we have bd r xy r b d u v...(1.10) r xy and r uv beng the coeffcent of correlaton between x and y and u and v respectvely, (1.10) establshed, numercally, the two correlaton coeffcents reman equal and they would have opposte sgns only when b and d, the two scales, dffer n sgn. () The coeffcent of correlaton always les between 1 and 1, ncludng both the lmtng values.e. 1 r 1...(1.11) Example 1. Compute the correlaton coeffcent between x and y from the followng data n 10, xy 0, x 00, y 6 x 40 and y 50 STATISTICS 1.9

10 CORRELATION AND REGRESSION Soluton From the gven data, we have by applyng (1.5), r n xy x y n x x n y y ( ) ( ) (40) 10 6 (50) Thus there s a good amount of postve correlaton between the two varables x and y. Alternately As gven, x 40 x 4 n 10 y 50 y 5 n 10 Cov (x, y) xy x. y n S x x (x) n COM M ON PROFICIENCY TEST

11 S y y y n 6? Thus applyng formula (1.1), we get r cov(x, y) Sx. S y As before, we draw the same concluson. Example 1.3 Fnd product moment correlaton coeffcent from the followng nformaton: Soluton X : Y : In order to fnd the covarance and the two standard devaton, we prepare the followng table: Table 1.3 Computaton of Correlaton Coeffcent x y x y x y (1) () (3) (1) x () (4) (1) (5) () STATISTICS 1.11

12 CORRELATION AND REGRESSION We have 9 x y cov (x, y) xy x y n 166/ (x) x n 163? (4.8333) S y y (y) n 79 (6.50) Thus the correlaton coeffcent between x and y n gven by r cov (x, y) S s x y We fnd a hgh degree of negatve correlaton between x and y. Also, we could have appled formula (1.5) as we have done for the frst problem of computng correlaton coeffcent. Sometmes, a change of orgn reduces the computatonal labor to a great extent. Ths we are gong to do n the next problem. 1.1 COM M ON PROFICIENCY TEST

13 Example 1.4 The followng data relate to the test scores obtaned by eght salesmen n an apttude test and ther daly sales n thousands of rupees: Salesman : Soluton scores : Sales : Let the scores and sales be denoted by x and y respectvely. We take a, orgn of x as the average of the two extreme values.e. 54 and 70. Hence a 6 smlarly, the orgn of y s taken as b Table 1.4 Computaton of Correlaton Coeffcent Between Test Scores and Sales. Scores Sales n u v u v u v (x ) Rs x 6 y 30 (1) (y ) () (3) (4) (5)(3)x(4) (6)(3) (7)(4) Total Snce correlaton coeffcent remans unchanged due to change of orgn, we have r r xy r uv n u v u v n u u n v v 8 90 ( 13) ( 14) 8 1 ( 13) 8 1 ( 14) STATISTICS 1.13

14 CORRELATION AND REGRESSION In some cases, there may be some confuson about selectng the par of varables for whch correlaton s wanted. Ths s explaned n the followng problem. Example 1.5 Examne whether there s any correlaton between age and blndness on the bass of the followng data: Age n years : No. of Persons (n thousands) : No. of blnd Persons : Soluton Let us denote the md-value of age n years as x and the number of blnd persons per lakh as y. Then as before, we compute correlaton coeffcent between x and y. Table 1.5 Computaton of correlaton between age and blndness Age n Md-value No. of No. of No. of xy x y years x Persons blnd blnd per () (5) () (5) (1) () ( 000) B lakh (6) (7) (8) P (4) yb/p 1 lakh (3) (5) Total COM M ON PROFICIENCY TEST

15 The correlaton coeffcent between age and blndness s gven by n xy x. y r n x ( x) n y ( y) (30) (150) whch exhbts a very hgh degree of postve correlaton between age and blndness. Example 1.6 Coeffcent of correlaton between x and y for 0 tems s 0.4. The AM s and SD s of x and y are known to be 1 and 15 and 3 and 4 respectvely. Later on, t was found that the par (0, 15) was wrongly taken as (15, 0). Fnd the correct value of the correlaton coeffcent. Soluton We are gven that n 0 and the orgnal r 0.4, x 1, y 15, S x 3 and S y 4 r cov (x, y) cov(x, y) 0.4 S S 3 4 x y Cov (x, y) 4.8 xy x y4.8 n xy xy 3696 Hence, corrected xy Also, S x 9 (x / 0) 1 9 x 3060 STATISTICS 1.15

16 CORRELATION AND REGRESSION Smlarly, S y 16 S y y y 480 Thus corrected x n x wrong x value + correct x value Smlarly corrected y Corrected x Corrected y Thus corrected value of the correlaton coeffcent by applyng formula (1.5) (45) (95) Example 1.7 Compute the coeffcent of correlaton between marks n Statstcs and Mathematcs for the bvarate frequency dstrbuton shown n table 1.1 Soluton For the sake of computatonal advantage, we effect a change of orgn and scale for both the varable x and y. Defne u x a x 10 b 4 And v j y c y 10 d 4 Where x and y j denote respectvely the md-values of the x-class nterval and y-class nterval respectvely. The followng table shows the necessary calculaton on the rght top corner of each cell, the product of the cell frequency, correspondng u value and the respectve v value has been shown. They add up n a partcular row or column to provde the value of f j u v j for that partcular row or column COM M ON PROFICIENCY TEST

17 Table 1.6 Computaton of Correlaton Coeffcent Between Marks of Mathematcs and Statstcs Class Interval Md-value Class Md V j f o f o u f o u f j u v j Interval -value u f oj f oj v j f oj v j f j u v j CHECK A sngle formula for computng correlaton coeffcent from bvarate frequency dstrbuton s gven by r N f u v f u f v,j j j o o j j o o oj j oj j N f u f u f v f v...( 1. 10) The value of r shown a good amount of postve correlaton between the marks n Statstcs and Mathematcs on the bass of the gven data. STATISTICS 1.17

18 CORRELATION AND REGRESSION Example 1.8 Gven that the correlaton coeffcent between x and y s 0.8, wrte down the correlaton coeffcent between u and v where () u + 3x and 4v + 16x () u 3x and 4v + 16x () u 3x and 4v 16x (v) u + 3x and 4v 16x Soluton Usng (1.10), we fnd that r xy bd b d ruv.e. r xy r uv f b and d are of same sgn and r uv r xy when b and d are of opposte sgns, b and d beng the scales of x and y respectvely. In (), u ( ) + (-3/) x and v ( 11/4) + ( 4)y. Snce b 3/ and d 4 are of same sgn, the correlaton coeffcent between u and v would be the same as that between x and y.e. r xy 0.8 r uv In (), u ( ) + (3/)x and v ( 11/4) + ( 4)y Hence b 3/ and d 4 are of opposte sgns and we have r uv r xy 0.8 Proceedng n a smlar manner, we have r uv 0.8 and 0.8 n () and (v). (c) SPEARMAN S RANK CORRELATION COEFFICIENT When we need fndng correlaton between two qualtatve characterstcs, say, beauty and ntellgence, we take recourse to usng rank correlaton coeffcent. Rank correlaton can also be appled to fnd the level of agreement (or dsagreement) between two judges so far as assessng a qualtatve characterstc s concerned. As compared to product moment correlaton coeffcent, rank correlaton coeffcent s easer to compute, t can also be advocated to get a frst hand mpresson about the correlaton between a par of varables. Spearman s rank correlaton coeffcent s gven by r R 1 6 d n(n 1)... (1.11) where r R denotes rank correlaton coeffcent and t les between 1 and 1 nclusve of these two values. d x y represents the dfference n ranks for the -th ndvdual and n denotes the number of ndvduals. In case u ndvduals receve the same rank, we descrbe t as a ted rank of length u. In case of a ted rank, formula (1.11) s changed to 1.18 COM M ON PROFICIENCY TEST

19 r R 3 tj tj 6 d + j (1.1) n n 1 In ths formula, t j represents the j th te length and the summaton (t 3 j t j) extends over the lengths of all the tes for both the seres. Example 1.9 compute the coeffcent of rank correlaton between sales and advertsement expressed n thousands of rupees from the followng data: Sales : Advertsement : Soluton Let the rank gven to sales be denoted by x and rank of advertsement be denoted by y. We note that snce the hghest sales as gven n the data, s 95, t s to be gven rank 1, the second hghest sales 90 s to be gven rank and fnally rank 8 goes to the lowest sales, namely 68. We have gven rank to the other varable advertsement n a smlar manner. Snce there are no tes, we apply formula (1.11). Table 1.7 Computaton of Rank correlaton between Sales and Advertsement. Sales Advertsement Rank for Rank for d x y d (x ) (y ) Sales (x ) Advertsement (y ) Total 0 4 j STATISTICS 1.19

20 CORRELATION AND REGRESSION Snce n 8 and d 4, applyng formula (1.11), we get. r R 1 6 d? n(n? 1) (8 1) The hgh postve value of the rank correlaton coeffcent ndcates that there s a very good amount of agreement between sales and advertsement. Example 1.10 Compute rank correlaton from the followng data relatng to ranks gven by two judges n a contest: Seral No. of Canddate : Rank by Judge A : Rank by Judge B : Soluton We drectly apply formula (1.11) as ranks are already gven. Table 1.8 Computaton of Rank Correlaton Coeffcent between the ranks gven by Judges Seral No. Rank by A (x ) Rank by B (y ) d x y d Total COM M ON PROFICIENCY TEST

21 The rank correlaton coeffcent s gven by r R 1 6 d n(n 1) (10 1) The very low value (almost 0) ndcates that there s hardly any agreement between the ranks gven by the two Judges n the contest. Example 1.11 Compute the coeffcent of rank correlaton between Eco. marks and stats. Marks as gven below: Soluton Eco Marks : Stats Marks : Ths s a case of ted ranks as more than one student share the same mark both for Economcs and Statstcs. For Eco. the student recevng 80 marks gets rank 1 one gettng 6 marks receves rank, the student wth 60 receves rank 3, student wth 56 marks gets rank 4 and snce there are two students, each gettng 50 marks, each would be recevng a common rank, the average of the next two ranks 5 and 6.e e and lastly the last rank.. 7 goes to the student gettng the lowest Eco marks. In a smlar manner, we award ranks to the students wth stats marks. Table 1.9 Computaton of Rank Correlaton Between Eco Marks and Stats Marks wth Ted Marks Eco Mark Stats Mark Rank for Eco Rank for Stats d x y d (x ) (y ) (x ) (y ) Total STATISTICS 1.1

22 CORRELATION AND REGRESSION For Economcs mark there s one te of length and for stats mark, there are two tes of lengths and 3 respectvely. Thus 3 ( tj tj) 1 ( 3 ) ( 3 ) ( 3 ) Thus r R ( ) 3 ( tj tj) 6 d + j 1 1 n n 1 6 ( ) 1 7(7 1) 0.15 Example 1.1 For a group of 8 students, the sum of squares of dfferences n ranks for Mathematcs and Statstcs marks was found to be 50 what s the value of rank correlaton coeffcent? Soluton As gven n 8 and d 50. Hence the rank correlaton coeffcent between marks n Mathematcs and Statstcs s gven by r R 1 6œ d n n 1 É (8 1) 0.40 Example 1.13 For a number of towns, the coeffcent of rank correlaton between the people lvng below the poverty lne and ncrease of populaton s If the sum of squares of the dfferences n ranks awarded to these factors s 8.50, fnd the number of towns. Soluton As gven r R 0.50, d Thus r R 1 6œ d n n 1 É 1. COM M ON PROFICIENCY TEST

23 n n ( ) n (n 1) 990 n (n 1) 10(10 1) n 10 as n must be a postve nteger. Example 1.14 Whle computng rank correlaton coeffcent between profts and nvestment for 10 years of a frm, the dfference n rank for a year was taken as 7 nstead of 5 by mstake and the value of rank correlaton coeffcent was computed as What would be the correct value of rank correlaton coeffcent after rectfyng the mstake? Soluton: We are gven that n 10, r R 0.80 and the wrong d 7 should be replaced by 5. r R œ d n n 1 É É 1 6œ d d 33 Corrected d Hence rectfed value of rank correlaton coeffcent ( ) 0.95 (d) COEFFICIENT OF CONCURRENT DEVIATIONS A very smple and casual method of fndng correlaton when we are not serous about the magntude of the two varables s the applcaton of concurrent devatons. Ths method nvolves n attachng a postve sgn for a x-value (except the frst) f ths value s more than the prevous value and assgnng a negatve value f ths value s less than the prevous value. Ths s done for the y-seres as well. The devaton n the x-value and the correspondng y-value s known to be concurrent f both the devatons have the same sgn. STATISTICS 1.3

24 CORRELATION AND REGRESSION Denotng the number of concurrent devaton by c and total number of devatons as m (whch must be one less than the number of pars of x and y values), the coeffcent of concurrent devaton s gven by r C + c m...(1.13) m If (c m) >0, then we take the postve sgn both nsde and outsde the radcal sgn and f (c m) <0, we are to consder the negatve sgn both nsde and outsde the radcal sgn. Lke Pearson s correlaton coeffcent and Spearman s rank correlaton coeffcent, the coeffcent of concurrent devatons also les between 1 and 1, both nclusve. Example 1.15 Fnd the coeffcent of concurrent devatons from the followng data. Year : Prce : Demand : Table 1.10 Soluton: Computaton of Coeffcent of Concurrent Devatons. Year Prce Sgn of Demand Sgn of Product of devaton devaton from devaton from the the prevous (ab) prevous fgure (b) fgure (a) In ths case, m number of pars of devatons 7 c No. of postve sgns n the product of devaton column Number of concurrent devatons 1.4 COM M ON PROFICIENCY TEST

25 Thus r C c m ± ± m 47 ± ± m 3 ± ± 7 (Snce c? m? 3 m we take negatve sgn both nsde and outsde of the radcal sgn) Thus there s a negatve correlaton between prce and demand. 1.5 REGRESSION ANALYSIS In regresson analyss, we are concerned wth the estmaton of one varable for a gven value of another varable (or for a gven set of values of a number of varables) on the bass of an average mathematcal relatonshp between the two varables (or a number of varables). Regresson analyss plays a very mportant role n the feld of every human actvty. A busnessman may be keen to know what would be hs estmated proft for a gven level of nvestment on the bass of the past records. Smlarly, an outgong student may lke to know her chance of gettng a frst class n the fnal Unversty Examnaton on the bass of her performance n the college selecton test. When there are two varables x and y and f y s nfluenced by x.e. f y depends on x, then we get a smple lnear regresson or smple regresson. y s known as dependent varable or regresson or explaned varable and x s known as ndependent varable or predctor or explanator. In the prevous examples snce proft depends on nvestment or performance n the Unversty Examnaton s dependent on the performance n the college selecton test, proft or performance n the Unversty Examnaton s the dependent varable and nvestment or performance n the selecton test s the In-dependent varable. In case of a smple regresson model f y depends on x, then the regresson lne of y on x n gven by y a + bx (1.14) Here a and b are two constants and they are also known as regresson parameters. Furthermore, b s also known as the regresson coeffcent of y on x and s also denoted by b yx. We may defne STATISTICS 1.5

26 CORRELATION AND REGRESSION the regresson lne of y on x as the lne of best ft obtaned by the method of least squares and used for estmatng the value of the dependent varable y for a known value of the ndependent varable x. The method of least squares nvolves n mnmzng e (y y ) (y a bx ). (1.15) where y demotes the actual or observed value and y a + b x, the estmated value of y for a gven value of x, e s the dfference between the observed value and the estmated value and e s techncally known as error or resdue. Ths summaton ntends over n pars of observatons of (x, y ). The lne of regresson of y or x and the errors of estmaton are shown n the followng fgure. y Regresson lne of y on x y a+bx e 1 >0 e 3 >0 e <0 e n < 0 0 x FIGURE 1.7 SHOWING REGRESSION LINE OF y ON x AND ERRORS OF ESTIMATION Mnmsaton of (1.15) yelds the followng equatons known as Normal Equatons. y na + b x.. (1.16) x y a x + b x..... (1.17) Solvng there two equatons for b and a, we have the least squares estmates of b and a as Cov(x, y) b S r.s x.s x y Sx 1.6 COM M ON PROFICIENCY TEST

27 r.s S x y...(1.18) After estmatng b, estmate of a s gven by ay bx... (1.19) Substtutng the estmates of b and a n (1.14), we get y y r x x S y S x...(1.0) There may be cases when the varable x depends on y and we may take the regresson lne of x on y as x a^+ b^y Unlke the mnmzaton of vertcal dstances n the scatter dagram as shown n fgure (1.7) for obtanng the estmates of a and b, n ths case we mnmze the horzontal dstances and get the followng normal equaton n a^ and b^, the two regresson parameters : x na^ + b^y... (1.1) x y a^y + b^ y..... (1.) or solvng these equatons, we get b^ b xy cov(x, y) r.s S S y y x...(1.3) ^? ^? anda x - b y... (1.4) A sngle formula for estmatng b s gven by n xy x. y ^ b b yx n y ( y )...(1.5) n xy x. y Smlarly, b^ b yx n y ( y )...(1.6) The standardzed form of the regresson equaton of x on y, as n (1.0), s gven by STATISTICS 1.7

28 CORRELATION AND REGRESSION x x r S x y y S y... (1.7) Example 1.15 Fnd the two regresson equatons from the followng data: x: y: Hence estmate y when x s 13 and estmate also x when y s 15. Soluton Table 1.11 Computaton of Regresson Equatons x y x y x y On the bass of the above table, we have x 34 x n 6 y 56 y n 6 cov (x, y)? xy? x y n x S x x n 1.8 COM M ON PROFICIENCY TEST

29 34 (5.6667) S y y y n 554? (9.3333) The regresson lne of y on x s gven by y a + bx cov(x, y) Where b^ S x ^ and a y? bx x Thus the estmated regresson equaton of y on x s y x When x 13, the estmated value of y s gven by ŷ The regresson lne of x on y s gven by x a^ + b^ y Where b^?? cov x, y S y STATISTICS 1.9

30 CORRELATION AND REGRESSION and a^ ? x b y Thus the estmated regresson lne of x on y s x y When y 15, the estmate value of x s gven by ˆx Example 1.16 Marks of 8 students n Mathematcs and statstcs are gven as: Mathematcs: Statstcs: Fnd the regresson lnes. When marks of a student n Mathematcs are 90, what are hs most lkely marks n statstcs? Soluton We denote the marks n Mathematcs and Statstcs by x and y respectvely. We are to fnd the regresson equaton of y on x and also of x or y. Lastly, we are to estmate y when x 90. For computaton advantage, we shft orgns of both x and y. Table 1.1 Computaton of regresson lnes Maths Stats u v u v mark (x ) mark (y ) x 74 y 76 u v COM M ON PROFICIENCY TEST

31 The regresson coeffcents b (or b yx ) and b (or b xy ) reman unchanged due to a shft of orgn. Applyng (1.5) and (1.6), we get n uv u. v b b yx b vu n u ( u ) 8.(71) (3).( 13) 8.(43) (3) n uv u. v and b^ b xy b uv n v ( v ) 8.(71) (3).( 13) 8.(559) ( 13) Also a ^ y b^ x (595) (595) ^ and a^ x? b y The regresson lne of y on x s y x and the regresson lne of x on y s x y STATISTICS 1.31

32 CORRELATION AND REGRESSION For x 90, the most lkely value of y s ŷ x Example 1.17 The followng data relate to the mean and SD of the prces of two shares n a stock Exchange: Share Mean (n Rs.) SD (n Rs.) Company A Company B Coeffcent of correlaton between the share prces 0.48 Fnd the most lkely prce of share A correspondng to a prce of Rs. 60 of share B and also the most lkely prce of share B for a prce of Rs. 50 of share A. Soluton Denotng the share prces of Company A and B respectvely by x and y, we are gven that x Rs. 44, y Rs. 58 S x Rs. 5.60, S y Rs and r 0.48 The regresson lne of y on x s gven by y a + bx Where b S r S y x a y bx Rs. ( ) Rs Thus the regresson lne of y on x.e. the regresson lne of prce of share B on that of share A s gven by y Rs. ( x) When x Rs. 50, Rs. ( ) 1.3 COM M ON PROFICIENCY TEST

33 Rs Agan the regresson lne of x on y s gven by x a^ + b^y The estmated prce of share B for a prce of Rs. 50 of share A s Rs Where b^ S r S x y a^? ^ x? b y Rs. ( ) Rs Hence the regresson lne of x on y.e. the regresson lne of prce of share A on that of share B n gven by x Rs. ( y) When y Rs. 60, ˆx Rs. ( ) Rs Example 1.18 The followng data relate the expendture or advertsement n thousands of rupees and the correspondng sales n lakhs of rupees. Soluton Expendture on Ad : Sales : Fnd an approprate regresson equaton. Snce sales (y) depend on advertsement (x), the approprate regresson equaton s of y on x.e. of sales on advertsement. We have, on the bass of the gven data, n 5, x y xy x b n y x y n x x STATISTICS 1.33

34 CORRELATION AND REGRESSION a y bx Thus, the regresson lne of y or x.e. the regresson lne of sales on advertsement s gven by y x 1.6 PROPERTIES OF REGRESSION LINES We consder the followng mportant propertes of regresson lnes: () The regresson coeffcents reman unchanged due to a shft of orgn but change due to a shft of scale. Ths property states that f the orgnal par of varables s (x, y) and f they are changed to the par (u, v) where x? a y? c u and v p q b yx q b vu p. (1.8) and bxy p b uv q (1.9) () The two lnes of regresson ntersect at the pont? x, y?, where x and y are the varables under consderaton. Accordng to ths property, the pont of ntersecton of the regresson lne of y on x and the regresson lne of x on y s? x, y?.e. the soluton of the smultaneous equatons n x and y. () The coeffcent of correlaton between two varables x and y n the smple geometrc 1.34 COM M ON PROFICIENCY TEST

35 mean of the two regresson coeffcents. The sgn of the correlaton coeffcent would be the common sgn of the two regresson coeffcents. Ths property says that f the two regresson coeffcents are denoted by b yx (b) and b xy (b ) then the coeffcent of correlaton s gven by r ± b b.. (1.30) yx xy If both the regresson coeffcents are negatve, r would be negatve and f both are postve, r would assume a postve value. Example 1.19 If the relatonshp between two varables x and u s u + 3x 10 and between two other varables y and v s y + 5v 5, and the regresson coeffcent of y on x s known as 0.80, what would be the regresson coeffcent of v on u? Soluton u + 3x 10 x10/3 u 1/3 and y + 5v 5 From v y5/ 5/ (1.8), we have q b yx b p vu or, 5/ 0.80 b 1/ b vu vu 8 b vu Example 1.0 For the varables x and y, the regresson equatons are gven as 7x 3y 18 0 and 4x y 11 0 () Fnd the arthmetc means of x and y. () Identfy the regresson equaton of y on x. STATISTICS 1.35

36 CORRELATION AND REGRESSION () Compute the correlaton coeffcent between x and y. (v) Gven the varance of x s 9, fnd the SD of y. Soluton () Snce the two lnes of regresson ntersect at the pont (x, y), replacng x and y by x and y respectvely n the gven regresson equatons, we get 7 x 3y 180 and 4 x? y? 110 Solvng these two equatons, we get x 3 and y 1 Thus the arthmetc means of x and y are gven by 3 and 1 respectvely. () Let us assume that 7x 3y 18 0 represents the regresson lne of y on x and 4x y 11 0 represents the regresson lne of x on y. Now 7x 3y y 6 + x 3 7 b yx 3 Agan 4x y x + y b xy Thus r b yx b xy < 1 1 Snce r 1 r 1, our assumptons are correct. Thus, 7x 3y 18 0 truly represents the regresson lne of y on x. () Snce r COM M ON PROFICIENCY TEST

37 r 7 1 (We take the sgn of r as postve snce both the regresson coeffcents are postve) (v) b yx S r S y x 7 3 S y ( S x 9 as gven) S y REVIEW OF CORRELATION AND REGRESSION ANALYSIS So far we have dscussed the dfferent measures of correlaton and also how to ft regresson lnes applyng the method of Least Squares. It s obvous that we take recourse to correlaton analyss when we are keen to know whether two varables under study are assocated or correlated and f correlated, what s the strength of correlaton. The best measure of correlaton s provded by Pearson s correlaton coeffcent. However, one severe lmtaton of ths correlaton coeffcent, as we have already dscussed, s that t s applcable only n case of a lnear relatonshp between the two varables. If two varables x and y are ndependent or uncorrelated then obvously the correlaton coeffcent between x and y s zero. However, the converse of ths statement s not necessarly true.e. f the correlaton coeffcent, due to Pearson, between two varables comes out to be zero, then we cannot conclude that the two varables are ndependent. All that we can conclude s that no lnear relatonshp exsts between the two varables. Ths, however, does not rule out the exstence of some non lnear relatonshp between the two varables. For example, f we consder the followng pars of values on two varables x and y. (, 4), ( 1, 1), (0, 0), (1, 1) and (, 4), then cov (x, y) ( + 4) + ( 1+1) + (0 0) + (1 1) + ( 4) 0 as x 0 Thus r xy 0 Ths does not mean that x and y are ndependent. In fact the relatonshp between x and y s y x. Thus t s always wser to draw a scatter dagram before reachng concluson about the exstence of correlaton between a par of varables. There are some cases when we may fnd a correlaton between two varables although the two varables are not causally related. Ths s due to the exstence of a thrd varable whch s related to both the varables under consderaton. Such a correlaton s known as spurous STATISTICS 1.37

38 CORRELATION AND REGRESSION correlaton or non-sense correlaton. As an example, there could be a postve correlaton between producton of rce and that of ron n Inda for the last twenty years due to the effect of a thrd varable tme on both these varables. It s necessary to elmnate the nfluence of the thrd varable before computng correlaton between the two orgnal varables. Correlaton coeffcent measurng a lnear relatonshp between the two varables ndcates the amount of varaton of one varable accounted for by the other varable. A better measure for ths purpose s provded by the square of the correlaton coeffcent, Known as coeffcent of determnaton. Ths can be nterpreted as the rato between the explaned varance to total varance.e. Explaned varance r Total varance Thus a value of 0.6 for r ndcates that (0.6) 100% or 36 per cent of the varaton has been accounted for by the factor under consderaton and the remanng 64 per cent varaton s due to other factors. The coeffcent of non-determnaton s gven by (1 r ) and can be nterpreted as the rato of unexplaned varance to the total varance. Coeffcent of non-determnaton (1 r ) Regresson analyss, as we have already seen, s concerned wth establshng a functonal relatonshp between two varables and usng ths relatonshp for makng future projecton. Ths can be appled, unlke correlaton for any type of relatonshp lnear as well as curvlnear. The two lnes of regresson concde.e. become dentcal when r 1 or 1 or n other words, there s a perfect negatve or postve correlaton between the two varables under dscusson. If r 0 Regresson lnes are perpendcular to each other COM M ON PROFICIENCY TEST

39 EXERCISE Set A Wrte the correct answers. Each queston carres 1 mark. 1. Bvarate Data are the data collected for (a) (b) (c) (d) Two varables More than two varables Two varables at the same pont of tme Two varables at dfferent ponts of tme.. For a bvarate frequency table havng (p + q) classfcaton the total number of cells s (a) p (b) p + q (c) q (d) pq 3. Some of the cell frequences n a bvarate frequency table may be (a) Negatve (b) Zero (c) a or b (d) Non of these 4. For a p x q bvarate frequency table, the maxmum number of margnal dstrbutons s (a) p (b) p + q (c) 1 (d) 5. For a p x q classfcaton of bvarate data, the maxmum number of condtonal dstrbutons s (a) p (b) p + q (c) pq (d) p or q 6. Correlaton analyss ams at (a) (b) (c) (d) Predctng one varable for a gven value of the other varable Establshng relaton between two varables Measurng the extent of relaton between two varables Both (b) and (c). 7. Regresson analyss s concerned wth (a) (b) (c) (d) Establshng a mathematcal relatonshp between two varables Measurng the extent of assocaton between two varables Predctng the value of the dependent varable for a gven value of the ndependent varable Both (a) and (c). STATISTICS 1.39

40 CORRELATION AND REGRESSION 8. What s spurous correlaton? (a) (b) (c) (d) It s a bad relaton between two varables. It s very low correlaton between two varables. It s the correlaton between two varables havng no causal relaton. It s a negatve correlaton. 9. Scatter dagram s consdered for measurng (a) (b) (c) (d) Lnear relatonshp between two varables Curvlnear relatonshp between two varables Nether (a) nor (b) Both (a) and (b). 10. If the plotted ponts n a scatter dagram le from upper left to lower rght, then the correlaton s (a) Postve (b) Zero (c) Negatve (d) None of these. 11. If the plotted ponts n a scatter dagram are evenly dstrbuted, then the correlaton s (a) Zero (b) Negatve (c) Postve (d) (a) or (b). 1. If all the plotted ponts n a scatter dagram le on a sngle lne, then the correlaton s (a) Perfect postve (b) Perfect negatve (c) Both (a) and (b) (d) Ether (a) or (b). 13. The correlaton between shoe-sze and ntellgence s (a) Zero (b) Postve (c) Negatve (d) None of these. 14. The correlaton between the speed of an automoble and the dstance travelled by t after applyng the brakes s (a) Negatve (b) Zero (c) Postve (d) None of these. 15. Scatter dagram helps us to (a) (b) (c) (d) Fnd the nature correlaton between two varables Compute the extent of correlaton between two varables Obtan the mathematcal relatonshp between two varables Both (a) and (c) COM M ON PROFICIENCY TEST

41 16. Pearson s correlaton coeffcent s used for fndng (a) (b) (c) (d) Correlaton for any type of relaton Correlaton for lnear relaton only Correlaton for curvlnear relaton only Both (b) and (c). 17. Product moment correlaton coeffcent s consdered for (a) (b) (c) (d) Fndng the nature of correlaton Fndng the amount of correlaton Both (a) and (b) Ether (a) and (b). 18. If the value of correlaton coeffcent s postve, then the ponts n a scatter dagram tend to cluster (a) (b) (c) (d) From lower left corner to upper rght corner From lower left corner to lower rght corner From lower rght corner to upper left corner From lower rght corner to upper rght corner. 19. When r 1, all the ponts n a scatter dagram would le (a) (b) (c) (d) On a straght lne drected from lower left to upper rght On a straght lne drected from upper left to lower rght On a straght lne Both (a) and (b). 0. Product moment correlaton coeffcent may be defned as the rato of (a) (b) (c) (d) The product of standard devatons of the two varables to the covarance between them The covarance between the varables to the product of the varances of them The covarance between the varables to the product of ther standard devatons Ether (b) or (c). 1. The covarance between two varables s (a) Strctly postve (b) Strctly negatve (c) Always 0 (d) Ether postve or negatve or zero.. The coeffcent of correlaton between two varables (a) (b) Can have any unt. Is expressed as the product of unts of the two varables STATISTICS 1.41

42 CORRELATION AND REGRESSION (c) (d) Is a unt free measure None of these. 3. What are the lmts of the correlaton coeffcent? (a) No lmt (b) 1 and 1 (c) 0 and 1, ncludng the lmts (d) 1 and 1, ncludng the lmts 4. In case the correlaton coeffcent between two varables s 1, the relatonshp between the two varables would be (a) y a + bx (b) y a + bx, b > 0 (c) y a + bx, b < 0 (d) y a + bx, both a and b beng postve. 5. If the relatonshp between two varables x and y n gven by x + 3y + 4 0, then the value of the correlaton coeffcent between x and y s (a) 0 (b) 1 (c) 1 (d) negatve. 6. For fndng correlaton between two attrbutes, we consder (a) (b) (c) (d) Pearson s correlaton coeffcent Scatter dagram Spearman s rank correlaton coeffcent Coeffcent of concurrent devatons. 7. For fndng the degree of agreement about beauty between two Judges n a Beauty Contest, we use (a) Scatter dagram (b) Coeffcent of rank correlaton (c) Coeffcent of correlaton (d) Coeffcent of concurrent devaton. 8. If there s a perfect dsagreement between the marks n Geography and Statstcs, then what would be the value of rank correlaton coeffcent? (a) Any value (b) Only 1 (c) Only 1 (d) (b) or (c) 9. When we are not concerned wth the magntude of the two varables under dscusson, we consder (a) Rank correlaton coeffcent (b) Product moment correlaton coeffcent (c) Coeffcent of concurrent devaton (d) (a) or (b) but not (c). 30. What s the quckest method to fnd correlaton between two varables? (a) Scatter dagram (b) Method of concurrent devaton (c) Method of rank correlaton (d) Method of product moment correlaton 1.4 COM M ON PROFICIENCY TEST

43 31. What are the lmts of the coeffcent of concurrent devatons? (a) (b) (c) (d) No lmt Between 1 and 0, ncludng the lmtng values Between 0 and 1, ncludng the lmtng values Between 1 and 1, the lmtng values nclusve 3. If there are two varables x and y, then the number of regresson equatons could be (a) 1 (b) (c) Any number (d) Snce Blood Pressure of a person depends on age, we need consder (a) (b) (c) (d) The regresson equaton of Blood Pressure on age The regresson equaton of age on Blood Pressure Both (a) and (b) Ether (a) or (b). 34. The method appled for dervng the regresson equatons s known as (a) Least squares (b) Concurrent devaton (c) Product moment (d) Normal equaton. 35. The dfference between the observed value and the estmated value n regresson analyss s known as (a) Error (b) Resdue (c) Devaton (d) (a) or (b). 36. The errors n case of regresson equatons are (a) Postve (b) Negatve (c) Zero (d) All these. 37. The regresson lne of y on x s derved by (a) (b) (c) (d) The mnmsaton of vertcal dstances n the scatter dagram The mnmsaton of horzontal dstances n the scatter dagram Both (a) and (b) (a) or (b). 38. The two lnes of regresson become dentcal when (a) r 1 (b) r 1 (c) r 0 (d) (a) or (b). 39. What are the lmts of the two regresson coeffcents? (a) No lmt (b) Must be postve STATISTICS 1.43

44 CORRELATION AND REGRESSION (c) (d) One postve and the other negatve Product of the regresson coeffcent must be numercally less than unty. 40. The regresson coeffcents reman unchanged due to a (a) Shft of orgn (b) Shft of scale (c) Both (a) and (b) (d) (a) or (b). 41. If the coeffcent of correlaton between two varables s 0 9, then the coeffcent of determnaton s (a) 0.9 (b) 0.81 (c) 0.1 (d) If the coeffcent of correlaton between two varables s 0.7 then the percentage of varaton unaccounted for s Set B (a) 70% (b) 30% (c) 51% (d) 49% Answer the followng questons by wrtng the correct answers. Each queston carres marks. 1. If for two varable x and y, the covarance, varance of x and varance of y are 40, 16 and 56 respectvely, what s the value of the correlaton coeffcent? (a) 0.01 (b) 0.65 (c) 0.4 (d) 0.5. If cov(x, y) 15, what restrctons should be put for the standard devatons of x and y? (a) No restrcton. (b) The product of the standard devatons should be more than 15. (c) The product of the standard devatons should be less than 15. (d) The sum of the standard devatons should be less than If the covarance between two varables s 0 and the varance of one of the varables s 16, what would be the varance of the other varable? (a) More than 100 (b) More than 10 (c) Less than 10 (d) More than If y a + bx, then what s the coeffcent of correlaton between x and y? (a) 1 (b) 1 (c) 1 or 1 accordng as b > 0 or b < 0 (d) none of these. 5. If r 0.6 then the coeffcent of non-determnaton s (a) 0.4 (b) 0.6 (c) 0.36 (d) COM M ON PROFICIENCY TEST

45 6. If u + 5x 6 and 3y 7v 0 and the correlaton coeffcent between x and y s 0.58 then what would be the correlaton coeffcent between u and v? (a) 0.58 (b) 0.58 (c) 0.84 (d) If the relaton between x and u s 3x + 4u and the correlaton coeffcent between x and y s 0.6, then what s the correlaton coeffcent between u and y? (a) 0.6 (b) 0.8 (c) 0.6 (d) From the followng data x: y: Two coeffcent of correlaton was found to be What s the correlaton between u and v as gven below? u: v: (a) 0.93 (b) 0.93 (c) 0.57 (d) Referrng to the data presented n Q. No. 8, what would be the correlaton between u and v? u: v: (a) 0.6 (b) 0.6 (c) 0.93 (d) If the sum of squares of dfference of ranks, gven by two judges A and B, of 8 students n 1, what s the value of rank correlaton coeffcent? (a) 0.7 (b) 0.65 (c) 0.75 (d) If the rank correlaton coeffcent between marks n management and mathematcs for a group of student n 0.6 and the sum of squares of the dfferences n ranks n 66, what s the number of students n the group? (a) 10 (b) 9 (c) 8 (d) Whle computng rank correlaton coeffcent between proft and nvestment for the last 6 years of a company the dfference n rank for a year was taken 3 nstead of 4. What s the rectfed rank correlaton coeffcent f t s known that the orgnal value of rank correlaton coeffcent was 0.4? (a) 0.3 (b) 0. (c) 0.5 (d) For 10 pars of observatons, No. of concurrent devatons was found to be 4. What s the value of the coeffcent of concurrent devaton? (a) 0. (b) 0. (c) 1/3 (d) 1/3 STATISTICS 1.45