Comments on Covariance, and Covariance Algebra John B. Willett

Transcription

1 Havad Uvety Gaduate School of Educato Commet o aace ad aace Algeba Joh B. llett Updated: 9/9/ 6:45 PM Etmatg the covaace of oe vaable wth aothe offe aothe mpotat way of ummazg the tegth of aocato betwee the two vaable. The oto of covaace betwee two vaable cloely elated to -- ad potetally moe ueful tha the oto of coelato wth whch you ae moe famla. I fact covaace the foudatoal quatty fom whch coelato deved. A coelato coeffcet mply the covaace betwee a pa of tadadzed vaable. I what follow I toduce ad defe the cocept of covaace ad I deve mple algebac ule by whch t ca be mapulated. It becaue we ca mapulate covaace algebacally th way that t the favoed alteatve to the moe famla cocept of coelato. A uual we coceve of covaace a a paamete that epeet the tegth of aocato betwee a pa of vaable ome defed populato. The we mage that we ca daw a ample fom that populato ad obta a ample etmate of the covaace coductg utable hypothe tet f we wat to make a feece back to the populato. Depte th ubqutou famg I have choe hee to defe ad mapulate ample etmate of covaace athe tha focu o the coepodg populato paamete. I have doe th becaue thee etmate ae ealy expeed ad mapulated ug the otato of ummato ymbol.e. otato whch I thk you wll be able to follow moe ealy. I a moe techcal peetato a a tadad coue pobablty ad tattc we would ft defe covaace a a populato paamete a the expectato of a co-poduct of vaable ad the ue tegal calculu to deve the ame covaace algeba that I peet below. Howeve a both the ample ad the populato appoach lead to the ame et of popete ad ule I have opted to peet the appoach that doe ot eque tegal calculu ad theefoe moe acceble pedagogcally. ou hould kow that tudyg th documet the ete pupoe of almot all of the page cludg th to -- to get you to the x Teet of aace Algeba lted out o page 5 though 8 patculaly the geeal ule o the lat page. It thee teet that we wll ue cla ad that I have foud o vey ueful thoughout my caee quattatve method. Thoughout the documet I have hghlghted the moe mpotat defto ad elatohp. Sample aace of Two Vaable The ample covaace of two cotuou vaable ad defed a a aveage of the copoduct of the vaable devato fom the ow mea a follow: hee ubcpt epeet the th dvdual the ample of ze ad ad epeet the ample mea of vaable ad epectvely a follow:

2 llett Page of 8 Mea ad Mea Notce that the covaace fomed by dvdg the um of the coepodg copoduct by - ad ot by the actual ample ze. ou wll ecall that th alo the cae wth the etmato of tadad devato ad vaace. The eao the ame both cae dvo by - athe tha eue that the ample covaace a ubaed etmate of populato covaace σ. hethe the deomato cota - o howeve t tll ueful to coceptualze covaace a a aveage co-poduct jut a t ueful to thk of tadad devato a a aveage dtace fom the mea. Udetadg how ample covaace ummaze the bvaate aocato betwee two vaable taghtfowad gaphcally a the two pael the fgue below. > Negatve Aocato Betwee & > Potve Aocato Betwee & > < I the fgue I peet tylzed ellptcal pot-cloud potayg bvaate elatohp betwee vaable ad. I the left-had pael the pot cloud exhbt a modeate potve aocato ad the ght had pael a modeate egatve aocato. I both pael I have added dotted vetcal ad hozotal cotucto le at the ample mea of both vaable. Thee dotted le effectvely dect the pot cloud to fou quadat. I the left pael whee the bvaate aocato of ad potve the uppe ght quadat the devato of ad fom the epectve mea ae both potve a the value of both vaable ae geate tha the epectve mea th quadat. Thu the co-poduct of the epectve devato mut alo be potve a they ae the the poduct of two potve umbe. I the lowe left quadat o the othe had the value of each of the vaable ae le tha the epectve mea ad o the devato fom thee mea ae egatve. But the all the copoduct th quadat tll ema potve a the poduct of two egatve umbe alo potve. Co-poduct the uppe-left ad lowe-ght quadat howeve ae egatve a they ae fomed by multplyg a potve ad a egatve devato togethe each cae.

3 llett Page 3 of 8 Becaue of the ged atue of each of the co-poduct all fou quadat the accumulato aco the ete ample ca be ethe potve o egatve o zeo! depedg o whethe the pot the ample pot cloud fall pedomatly the uppe-ght/lowe-left o uppe-left/lowe-ght quadat. I the left-had pael of Fgue fo tace whch potay a potve aocato the pot cloud oeted fom lowe-left to uppe-ght ad o the datapot ae dtbuted pedomatly the uppe-ght ad lowe-left quadat. Th mea that the ample co-poduct wll be potve pedomatly ad the accumulato wll total to a potve um. Thu the covaace whch the aveage of all of the co-poduct -- wll telf be potve. he the tegth of the exhbted potve aocato geate the pot cloud wll ema ellptcal but become aowe taveally. Coequetly the data-pot the cloud wll become cocetated ceagly the uppe-ght ad lowe-left quadat ad fewe wll be located the othe two quadat. The coepodg co-poduct wll be potve pedomatly ad the aveage value that t the covaace wll become ceagly lage ad potve. Thee o lmt to the actual magtude of the covaace t doe ot have a uppe boud of fo tace lke the coelato coeffcet becaue the ze of the cotbutg co-poduct lmted oly by the cale of the vaable beg mapulated. A you chage the cale of ad the magtude of the value chage a doe the magtude of the co-poduct ad the aveage the covaace. he the dplayed bvaate aocato egatve ee ght-had pael the agumet detcal. Howeve the pot-cloud ow oeted dowwad fom the uppe-left to the loweght quadat ad the co-poduct fomed the pot cloud ae egatve pedomatly povdg a egatve value fo the aveage the covaace. Fally whe thee o aocato at all betwee ad the pot cloud wll become ccula athe tha ellptcal ad the data-pot wll be dtbuted equally thoughout all fou quadat. That mea thee wll be oughly equal umbe of both potve ad egatve co-poduct the oveall um the value wll the ted to cacel each othe out ad the covaace wll ted to zeo a atcpated. aace ad Coelato ou may be tuck by the mlaty betwee the umeato of the defto of ample covaace ad the uual defto of the ample coelato coeffcet: Co 3 Thee a much deepe coceptual mlaty howeve that evealed by dvdg both the umeato ad deomato of the expeo o the ght had de of 3 by - to gve: 4

4 llett Page 4 of 8 The eplacg the expeo the umeato fom the defto of covaace we ee that the coelato coeffcet equal to the covaace of ad dvded by the poduct of the tadad devato o: hee a uual: 5 Vaace ad Vaace 6 So the ample coelato of ad jut the ample covaace of the two vaable dvded by the poduct of the ample tadad devato. Aothe way of gag ght to how covaace ummaze bvaate aocato betwee two vaable ca be obtaed by e-expeg the deomato 4 tem of the epectve tadad devato of ad a follow: hch ca be ewtte a: 8 Ad o compag the coelato coeffcet 8 wth the defto of covaace we ee that the coelato of ad mply the covaace of the tadadzed veo of vaable ad a I oted my toducto. To um up coelato ad covaace each meaue ad ummaze the tegth of bvaate aocato but the coelato a cale-fee meaue whch age betwee - ad whle the covaace a cale-depedet meaue whch ca age betwee egatve ad potve umbe of abtay magtude depedg o the metc of the cottuet vaable. The calefee atue of the coelato ca be vey ueful f you wat to compae the tegth of bvaate aocato aco vaable ad ettg. But fomato about the cale of the cottuet vaable that the epectve metc ha bee acfced dug the tadadzato poce that poduced the coelato coeffcet. aace o the othe had eta ad copoate the calg of t cottuet vaable ad theefoe cota addtoal fomato that become ueful whe oe wat to do moe tha mply chat about the tegth of aocato a we wll ee below. Oe teetg featue of the ubqutou bvaate omal dtbuto that the ample mea vecto whch cota the mea of both ad ad the ample covaace matx whch cota the vaace of ad alog wth the covaace togethe povde a et of tattc 7

5 llett Page 5 of 8 that ae uffcet fo decbg the ete hape of the dtbuto. Becaue of th mot of the omal-theoy aalytc method wth whch you ae famla cludg multple egeo aaly pcpal compoet aaly eve tuctual equato modelg ad othe ca be executed dectly fom the mea vecto ad the covaace matx whch ae kow a the uffcet tattc athe tha fom the aw data themelve! Th oe eao that joual eque autho to povde appedce that cota the ample mea vecto ad covaace matx of the vaable cluded the aalye. aace ad the Lea Regeo of o Thee aothe teetg quatty that coected tmately wth covaace the lope paamete a mple oday leat-quae OLS lea egeo of o. Recall that uch a aaly the hypothezed tattcal model : β β ε 9 th the uual otato. Ug the method of oday leat-quae we kow that the OLS ample etmate of the o the lope paamete a follow: ˆ β ˆ β Aga dvdg both the umeato ad deomato of the expeo o the ght had de of by - we fd that the OLS etmate of the o egeo lope : O ˆ β ˆ β Ad o teetgly the OLS-etmate of the lope of the o egeo mply the ample covaace of ad dvded by the vaace of. Teet of aace Algeba A mpotat attbute of the oto of covaace that oe ca deve a mple et of algebac ule fo t mapulato ethe the ample o the populato. I deve thee ule below begg wth the mplet. #: aace of a Vaable ad a Cotat I eo Poof. Image that the value of the ample cotat ad equal to cotat k fo each dvdual. The of coue the aveage value of wll alo be equal to k. So ubttutg to we fd that the covaace of ad k :

6 k k k k llett Page 6 of 8 Th make ee tutvely of coue becaue whe the value of a vaable lke ae cotat that whe doe ot vay the t caot co-vay wth aythg ele! #: aace of a Vaable wth Itelf I It Vaace Poof. Aothe way of ayg the ame thg to ay that the vaace of a vaable t covaace wth telf. Subttutg to the covaace of wth telf : Va 3 #3: Multplyg a Vaable by a Cotat Multple It aace wth aothe Vaable by the Same Cotat Poof. Image that each dvdual the ample ha the value of multpled by cotat k. The of coue the aveage value of wll alo be multpled by k. So ubttutg to the covaace of ad k wll become: k k k k k k k k k 4 Aga th make ee tutvely becaue multplyg a vaable by a cotat equvalet to ecalg t lke covetg a meaue of heght fom feet to che ay. Ad becaue covaace cale-depedet t value mut be mpacted by the e-calg f you multply heght by the t covaace wth omethg ele lke weght wll alo be multpled by! #4: aace of Oe Vaable wth the Sum of Two Othe Vaable I Equal to the Sum of the aace of the Ogal Vaable wth Each of the Othe Two Vaable Sepaately Poof. Th dffcult to ay but alo make ee. Let mage that vaable equal to the um of two othe vaable ad. The the aveage value of wll be the um of the aveage value of ad. Subttutg to the covaace of wth the:

7 llett Page 7 of 8 5 #5: aace of Oe Vaable wth the Dffeece of Two Othe Vaable I Equal to the Dffeece of the aace of the Ogal Vaable wth Each of the Othe Two Vaable Sepaately Poof. Th detcal to the pevou poof wth the um of ad beg eplaced by the dffeece. All that happe that oe of the g get chaged. Subttutg to the covaace of wth - : { } { } 6 #6: Geeal Rule fo the Algebac Mapulato of aace It may ot be eceay to cotue wth thee poof ay futhe tha th becaue you may have deduced the dect pcple by ow. If you pect the ule 5 ad 6 fo tace you may be tuck by the mlaty wth the uual ule fo multplyg out tadad algebac expeo. Fo tace compae the followg two equato: 7 Ad the followg:

8 llett Page 8 of 8 Notce that the covaace eult ca be obtaed o at leat emembeed -- by multplyg out the expeo wth the covaace paethee o the left had de of the equal g ug tadad algebac pcple ad the mply etg a opeato fot of whateve poduct you ed up wth. Thu the covaace of wth mply the covaace of wth plu the covaace of wth. The covaace of wth - jut the covaace of wth mu the covaace of wth ad o o. It exactly lke algebac multplcato but wth a few toed fo good meaue. It eay to how by the ummato method above that th covaace algeba wok fo computg the covaace of ay lea combato of vaable wth ay othe lea combato of vaable a follow: 8 8 It eve wok wth weghted lea combato of vaable a follow: Notce that I have take advatage of Rule #3 to move the weght whch ae cotat though the opeato ad have multpled them togethe a appopate each cae. Thee Rule Of aace Algeba ca be vey ueful fgug out a vaety of thg cludg: a why ubaed etmato of lope OLS egeo aaly eque a aumpto that the pedcto ad edual be ucoelated o b the atue of tumetalvaable etmato. Bewae howeve! The ule that I have lad out above apply oly to the covaace of weghted lea combato of vaable ad cotat. They do ot wok fo combato that ae multplcatve o that volve ato of vaable uch a the covaace of ad o the vaace of / fo tace. ou eed a much moe ophtcated et of ule fo that ad ome of them ae oly appoxmate. Cocluo Ty thee example out fo youelf fo fu. Ue covaace algeba to pove:. Va 3 9Va.. Va Va. 3. Va Va Va. 4. Va Va Va. 5. β β β Va. 9