x f(x) x

Transcription

1 BIVARIATE DISTRIBUTIONS Let be a varable that assumes the values { 1,,..., n }. Then, a functon that epresses the relatve frequenc of these values s called a unvarate frequenc functon. It must be true that f( ) 0 for all and The followng table provdes a trval eample: f() f( )1. Let and be varables that assume values n the the sets { 1,,..., n } and { 1,,..., m }, respectvel. Then the functon f(, j ), whch gves the relatve frequences of the occurrence of the pars (, j ) s called a bvarate frequenc functon. It must be true that f(, j ) 0 for all and An eample of a bvarate frequenc table s as follows: f(, j )1. The values of f(, j ) are wthn the bod of the table. The margnal frequenc functon of gves the relatve frequences of the values of regardless of the values of j wth whch the are assocated; and t s defned b f( ) f(, j ); 1,...,n. j j It follows that f( ) 0, and f( ) f(, j )1, j The margnal frequenc functon f( j ) s defned analogousl. The bvarate frequenc table above provdes eamples of the two margnal frequenc functons: f( 1) , f( 1)

2 and f( 1) , f( 0) , f( 1) The condtonal frequenc functon of gven j gves the relatve frequenc of the values of n the subset of f(, ) for whch j ; and t s gven b f( j ) f(, j ). f( j ) Observe that f( j ) f(, j ) f( j ) An eample based on the bvarate table s as follows: f( ) f( j) f( j ) 1. f( 1) f( 0) f( 1) (0.04/0.16) 0.5 (0.01/0.04) 0.5 (0.0/0.80) (0.1/0.16) 0.75 (0.03/0.04) 0.75 (0.60/0.80) We ma sa that s ndependent of f and onl f the condtonal dstrbuton of s the same for all values of, as t s n ths table. The condtonal frequenc functons of are the same for all values of f and onl f the are all equal to the margnal frequenc functon of. Proof. Suppose that f ( )f( 1 ) f( m ). Then f( ) j f( j )f( j )f ( ) j f( j )f ( ), whch s to sa that f ( )f( ), Conversel, f the condtonals are all equal to the margnal, then the must be equal to each other. Also observe that, f f( j )f( ) for all j and f( j )f( j ) for all, then, equvalentl, f(, j )f( j )f( j )f( j )f( )f( )f( j ). The condton that f(, j )f( )f( j ) consttutes an equvalent defnton of the ndependence of and. We have been concerned, so far, wth frequenc functons. These provde the prototpes for bvarate probablt mass functons and for bvarate probablt denst functons. The etenson to probablt mass functons s mmedate. For the case of the denst functons, we consder a two-dmensonal space R whch s defned as the set of all ordered pars (, ); <,<, whch correspond to the co-ordnates of the ponts n a plane of nfnte etent.

3 We suppose that there s a probablt measure defned over R such that, for an A R, P (A) s the probablt that (, ) falls n A. Thus, for eample, f A {a < b, a < b}, whch s a rectangle n the plane, then P (A) d c { b a } f(, )d d. Ths s a double ntegral, whch s performed n respect of the two varables n successon and n ether order. Usuall, the braces are omtted, whch s allowable f care s taken to ensure the correct correspondence between the ntegral sgns and the dfferentals. Eample. Let (, ) be a random vector wth a p.d.f of f(, ) 1 (6 ); 0 ; 4. 8 It needs to be confrmed that ths does ntegrate to unt over the specfed range of (, ). There s 1 4 )dd 8 0 (6 1 ] 4 [6 d (6 ) d 1 [6 ] Moments of a bvarate dstrbuton. Let (, ) have the p.d.f. f(, ). Then, the epected value of s defned b E() f(, )dd f()d, f s contnuous, and b E() f(, ) f(), f s dscrete. Jont moments of and can also be defned. For eample, there s ( a) r ( b) s f(, )dd, where r, s are ntegers and a, b are fed data. The most mportant jont moment for present purposes s the covarance of and, defned b C(, ) { E()}{ E()}f(, )dd. If, are statstcall ndependent, such that f(, ) f()f(), then ther jont moments can be epressed as the products of ther separate moments. 3

4 Thus, for eample, f, are ndependent then E{[ E()] [ E()] } [ E()] [ E()] f()f()dd [ E()] f()d [ E()] f()d V ()V (). The case of the covarance of,, when these are ndependent, s of prme mportance: C(, ) E{[ E()][ E()]} [ E()]f()d [ E()]f()d {[E() E()][E() E()]} 0. These relatonshps are best epressed usng the notaton of the epectatons operator. Thus C(, ) E{[ E()][ E()]} E[ E() E()+E()E()] E() E()E() E()E() + E()E() E() E()E(). Snce E() E()E() f, are ndependent, t follows, n that case, that C(, ) 0. Observe also that C(, ) E{[ E()] } V (). Now consder the varance of the sum +. Ths s V ( + ) E { [( + ) E( + )] } E { [{ E()} + { E()}] } E { [ E()] +[ E()] +[ E()][ E()] } V ()+V ()+C(, ). If, are ndependent, then C(, ) 0 and V ( + ) V () +V (). mportant to note that If, are ndependent, then the covarance s C(, ) 0. However, the condton C(, ) 0does not, n general, mpl that, are ndependent. It s A partcular case n whch C(, ) 0 does mpl the ndependence of, s when both these varables are normall dstrbuted. The correlaton coeffcent. To measure the relatedness of and, we use the correlaton coeffcent, defned b Corr(, ) C(, ) V ()V () E{[ E()][ E()]}) E{[ E()] }E{[ E()] }. Notce that ths s a number wthout unts. It can be shown that 1 Corr(, ) 1. If Corr(, ) 1, then there s a perfect postve correlaton between and, whch means that the le on a straght lne of postve slope. If Corr(, ) 1, then there s a perfect negatve correlaton; and the straght lne has a negatve slope. In other cases, there s a 4

5 scatter of ponts n the plane; and, f Corr(, ), then there s no lnear relatonshp between and. These results concernng the range of the correlatson coffcent follow from a verson of the Cauch Schwarz nequalt, whch wll be establshed at the end of the net secton. REGRESSION AND CONDITIONAL EXPECTATIONS Lnear condtonal epectatons. If, are correlated, then a knowledge of one of them enables us to make a better predcton of the other. Ths knowledge can be used n formng condtonal epectatons. In some cases, t s reasonable to make the assumpton that the condtonal epectaton E( ) s a lnear functon of : E( ) α + β. () Ths functon s descrbed as a lnear regresson equaton. The error from predctng b ts condtonal epectaton can be denoted b ε E( ); and therefore we have E( )+ε α + β + ε. Our object s to epress the parameters α and β as functons of the moments of the jont probablt dstrbuton of and. Usuall, the moments of the dstrbuton can be estmated n a straghtforward wa from a set of observatons on and. Usng the relatonshp that ets between the parameters and the theoretcal moments, we should be able to fnd estmates for α and β correspondng to the estmated moments. We begn b multplng equaton () throughout b f(), and b ntegratng wth respect to. Ths gves the equaton E() α + βe(), () whence α E() βe(). These equatons shows that the regresson lne passes through the pont E(, ) {E(),E()} whch s the epected value of the jont dstrbuton. B puttng () nto (), we fnd that E( ) E()+β { E() }, whch shows how the condtonal epectaton of dffers from the uncondtonal epectaton n proporton to the error of predctng b takng ts epected value. Now let us multpl () b and f() and then ntegrate wth respect to to provde E() αe()+βe( ). (v) Multplng () b E() gves () E()E() αe()+β { E() }, (v) 5

6 whence, on takng (v) from (v), we get [ E() E()E() β E( ) { E() } ], whch mples that β E() E()E() E( ) { E() } [ { }{ } ] E E() E() [ { } ] E E() C(, ) V (). (v) Thus, we have epressed α and β n terms of the moments E(), E(), V () and C(, ) of the jont dstrbuton of and. It should be recognsed that the predcton error ε E( ) α β s uncorrelated wth the varable. Ths s shown b wrtng [ { } ] E E( ) E() αe() βe( )0, (v) where the fnal equalt comes from (v). Ths result s readl ntellgble; for, f the predcton error were correlated wth the value of, then we should not be usng the nformaton of effcentl n predctng. Ths secton ma be concluded b provng a verson of the Cauch Schwarz nequalt that establshes the bounds on Corr(, ) C(, )/ V ()V (), whch s the coeffcent of the correlaton of and. Consder the varance of the predcton error ( [{ ] ) E E()} β{ E()} V () βc(, )+β V () 0. Settng β C(, )/V () gves {C(, )} V () V () + {C(, )} V () 0. whence V ()V () {C(, )}. It follows that {Corr(, )} 1 and, therefore, that 1 Corr(, ) 1. 6

7 Emprcal Regressons. Imagne that we have a sample of T observatons on and whch are ( 1, 1 ), (, ),...,( T, T ). Then we can calculate the followng emprcal or sample moments: 1 T ȳ 1 T S 1 T S 1 T t, t, ( t ) 1 T ( t ) t 1 T ( t )( t ȳ) 1 T t, ( t ) t 1 T t t ȳ, It seems reasonable that, n order to estmate α and β, we should replace the moments n the formulae of () and (v) b the correspondng sample moments. Thus the estmates of α and β are ˆα ȳ ˆβ, (t )( t ȳ) ˆβ (t ). The justfcaton of ths estmaton procedure, whch s know as the method of moments, s that, n man of the crcumstances under whch the sample s lable to be generated, we can epect the sample moments to converge to the true moments of the bvarate dstrbuton, thereb causng the estmates of the parameters to converge lkewse to ther true values. Often there s nsuffcent statstcal regulart n the processes generatng the varable to justf our postulatng a jont probablt denst functon for and. Sometmes the varable s regulated n pursut of an economc polc n such a wa that t cannot be regarded as random n an of the senses accepted b statstcal theor. In such cases, we ma prefer to derve the estmators of the parameters α and β b methods whch make fewer statstcal assumptons about. When s a non stochastc varable, the equaton α + β + ε s usuall regarded as a functonal relatonshp between and that s subject to the effects of a random dsturbance term ε. It s commonl assumed that, n all nstances of ths relatonshp, the dsturbance has a zero epected value and a varance whch s fnte and constant. Thus E(ε) 0 and V (ε) E(ε )σ. Also t s assumed that the movements n are unrelated to those of the dsturbance term. 7

8 The prncple of least squares suggests that we should estmate α and β b fndng the values whch mnmse the quantt S ( t ŷ t ) ( t α t β). Ths s the sum of squares of the vertcal dstances measured parallel to the - as of the data ponts from an nterpolated regresson lne. Dfferentatng the functon S wth respect to α and settng the results to zero for a mnmum gves ( t α β t )0, or, equvalentl, ȳ α β 0. Ths generates the followng estmatng equaton for α: α(β) ȳ β. (v) Net, b dfferentatng wth respect to β and settng the result to zero, we get t ( t α β t )0. () On substtutng for α from (v) and elmnatng the factor, ths becomes t t t (ȳ β ) β t 0, whence we get ˆβ t t T ȳ t T (t )( t ȳ) (t ). Ths epresson s dentcal to the one that we have derved b the method of moments. B puttng ˆβ nto the estmatng equaton for α under (v), we derve the same estmate ˆα for the ntercept parameter as the one obtaned b the method of moments. It s notable that the equaton () s the emprcal analogue of the equaton (v) whch epresses the condton that the predcton error s uncorrelated wth the values of. The method of least squares does not automatcall provde an estmate of σ E(ε t ). To obtan an estmate, we ma nvoke the method of moments whch, n vew of the fact that the regresson resduals e t t ˆα ˆβ t represent estmates of the correspondng values of ε t, suggests an estmator n the form of σ 1 T e t. In fact, ths s a based estmator wth E ( T σ ) { T } σ ; so t s common to adopt the unbased estmator e ˆσ t T. 8