Yield Analysis and Mixed Model

Transcription

1 ASA Secton on Nonpaaetc Statstcs Yeld Analyss and Med Model Eugene Dedeno and Igo Mandel Datouth College, 797 Rubn, DHMC, Lebanon, NH 3756 Meda Plannng Goup, 95 Boadway, New Yo, NY 7 go.andel@pg.co Abstact A concept of yeld analyss, an appoach fo estatng an obsevaton-specfc pedcton of the dependent vaable, has been developed. A elatonshp wth Hldetch-Houc egesson wth ando coeffcents s establshed. Two types of estaton ae consdeed: () dstbuton-fee that s based on vaance least squaes and weghted least squaes and () au lelhood that uses noal assupton; not noally dstbuted egesson coeffcents ae consdeed as well. Afte fed effect coeffcents ae estated, obsevatonspecfc pedctons ae coputed as posteo eans usng the BLUP appoach. The ed odel ples the yeld odel when the vaance of ando coeffcents goes to nfnty and the desgn at s othogonal. Monte Calo sulaton esults ae pesented povng odels hgh effcency. Case study esults ae shown, whee the applcaton of the poposed technque to a ca advetsng data deonstates a hgh pedctablty whle standad egesson fals wth wea coelatons between cost of ad and ca sales. Keywods: ando coeffcents, Hldetch-Houc egesson, ed odel, vaance least squaes, Monte Calo sulaton. Intoducton Lnea egesson wth ando coeffcents s a developed statstcal topc. Pehaps the ealest authos ae (Hldeth and Houc 968) and (Sway 97). A copehensve account of avalable estaton ethods was pesented n (Ra 975); the latest esults fo aal lelhood estaton ae obtaned n (Zaan ). On the othe hand, the concept of yeld analyss, an appoach whee the an goal s to estate ndvdual coeffcents of nfluence ( yelds ) on each obect of the populaton, was pesented (wth heustc estaton algoth) n (Mandel 3) and appled to tes sees pobles n (Mandel and Hause 5,). Hee we ege the two appoaches and stess that egesson wth ando coeffcents ay be vewed as a ed effects odel whee the sze of the cluste s one. As such the theoy of egesson wth ando coeffcents ay be futhe advanced usng ecent developents n ed odels (Dedeno 4). Whle pevous wos on ando coeffcents was concentated on estaton of fed effects, we focus on estaton of the ando effects (yelds) the ost nteestng pat fo the applcaton pont of vew, whch also have a geat potance n any f not all egesson-le applcatons. In the second pat of ths atcle we gve a eal lfe eaple, deonstatng a typcal poble whch ay occu n analyss and cannot be solved wth tadtonal eans; n pats 3-5 all of the an esults and poofs ae gven; pat 6 descbes esults the of a Monte Calo sulaton; pat 7 gves detals of pactcal pleentaton and ecoendatons; the concluson suazes esults and ponts out the pospectve aeas fo use and developent of the appoach.. Motvatng Eaple Copanes spend bllons of dollas on advetsng. Does advetsng ncease sales? We show the scatte-plot of sales vesus cost of advetsng of a lage ca anufactung copany n Fg.. The coelaton coeffcent.9 assues a negatve effect of advetsng to sales (!); aound 6% sales should be eplaned by that advetsng. Thus, accodng to the standad statstcal test the answe s "Not eally." Why do busnessen spend oney n van? It s possble, n pncple, but not eally plausble on a lage scale - contovesal pctues le pesented on a Fg. ae not unusual fo any stuatons. What ay be wong? Indeed, the affect of advetsng on sales s a coplcated pocess. Many thngs ae gong on, such as advetseent of the sae type of poduct by copettos, bad econocal stuaton and so on. The hypothess that the ate at whch ads affect sales s not constant ove te and vaes wthn a wee o even a day loos not only possble, but, n fact, the only vable. And ths s eactly what we ty to captue and what ay eplan pctue le that and any othes. 636

2 ASA Secton on Nonpaaetc Statstcs Sales Fgue. Relaton between advetsng spendng and ca sales n Honolulu (scales ae condtonal), coelaton ˆ β ( X S X ) X S y, (3) whee S n s a at, whee dagonal eleents ae and the off-dagonal eleents ae zeoes ,.,. 3,. 4,. 5,. Tadtonal egesson-le odels fal n dong that. To account fo uncontollable factos ando effects ay be ntoduced that leads to egesson wth ando coeffcents, but wth estaton of those on each obect (o oent of te). Then the possble eplanaton wll be, that advetsng wos, but t s effcency changes ove te n such a way, that bgge spendng bng less pe dolla spent (stll postvely affectng sales). 3. Regesson Model wth Rando Coeffcents In ths secton we descbe how yelds can be studed va egesson odel wth ando coeffcents, naely, y β + b b + ε whee foally.we can cobne the ntecept β wth egesson eo ε, so ntecept ay be teated ando as well: b β + δ,..., b b + δ whee δ ae ndependent and ( δ ), E( δ ) E o y β + β β + u, () whee E u ), E( u ) ( z, y b Thoughout the pape we denote ; when const we have, whee denotes vecto, X - at. Unde noal assupton y ~N ( β, ),,,... n. () Spendng Paaetes ae called vaance paaetes. If they ae nown we apply weghted least squaes 4. Methods of Estaton It s nteestng to now the lt of the GLS estato when one of the vaances goes to nfnty. Lets pc fo,,, and put to nfnty so that othe vaances stay bounded, so that l ( / ). Lettng vaances go to nfnty we obtan ~ ˆ β l β ( X Z X ) X Z y, whee Z dag(,,..., n ). Two types of estaton ethods ae dscussed n ths secton. The fst type, whch we call vaance least squaes, does not assue a dstbuton fo ando coeffcents and theefoe s dstbuton fee. The second type s au lelhood (ML) estaton that assues that coeffcents ae noally dstbuted. We dscuss standad and estcted ML estaton. 4. Vaance Least Squaes The odnay least squaes (OLS) estato ˆ β ( X X ) X y s an unbased and OLS consstent estato of β.theefoe, at least fo lage saple, we ay epect that the OLS esdual uˆ y βˆ OLS s a satsfactoy estate of the eo te u n egesson (). Consequently, uˆ ay be teated as an epcal estate of vaanceu whch s, the theoetcal vaance. Hence we can fnd an estate fo the vaance paaetes whch nzes the Eucldean dstance between the epcal and theoetcal vaances as n ( uˆ ). (4) Appaently, (4) s the su of squaes n egesson of squaed esduals on squaed eplanatoy vaables wth the least squaes soluton 637

3 ASA Secton on Nonpaaetc Statstcs ˆ & & & ( X X ) X uˆ & VLS, (5) whee, followng the notaton of (Hldeth and Houc 968) and (Ra 975), we use a dot ove the vecto o at to ndcate that the eleents ae squaed. Ths estato was deved by the fst authos - we call t vaance least squaes (VLS) estato because ths nae s self-eplanatoy (also ths ethod of vaance estaton was used n (Dedeno 4) n a oe geneal settng). The VLS estato s based because the epected value of uˆ s not eactly equal to. To fnd the unbased VLS estato we obseve that the covaance at of the OLS esdual vecto s u ˆ) cov( PSP, whee P I H and H X ( X X ) X s the poecton o hat at. Mat H eeges n connecton wth outles detecton; the dagonal eleent of ths at s the called leveage value (Hube, 98). The su of squaed eleents of the dffeence between epcal and theoetcal covaance atces taes the fo ˆ t( uuˆ PSP), whch s nzed ove. Afte soe algeba we ave at the unbased VLS (UVLS) estato ˆ & & & & & UVLS ( X PX ) X uˆ. (6) One ay teate and afte vaance paaetes ae estated plug the n (3) to obtan new esduals and contnue untl convegence. Ths type of estaton s called teatve VLS. 4. Mau Lelhood Now we assue that ando coeffcents ae noally dstbuted so that the odel s wtten as (). The log-lelhood functon, up to a constant te s gven by l ( β, ) ( y + β ) ln( ) (7) Then f s held fed l s azed at the weghted least squaes wth the weghts w / o n at fo (3). Thee s no closed fo soluton fo nzaton ove when β s held fed, so we need to do teatons. Fo ths we copute the fst and second devatves of the log-lelhood functon: l n e + (8) ( ) and the Hessan wth the (, ) th eleent l n. 3 ( ) ( ) e We tae the epectaton of the Hessan to copute the Fshe nfoaton at. Afte soe algeba we obtan that the nfoaton at fo v s gven by n I,,,,.., H ( ) ence, the Fshe scong algoth fo the vaance paaetes taes the fo l s+ s ( I ), s,,... Note that the nfoaton at fo β and s bloc-dagonal whch eans that () the loglelhood ay be azed altenatng between β and azaton, () the MLE fo coeffcents and vaance paaetes ae ndependent, () any consstent estato of leads to an effcent estato of β n the estated weghted least squaes. The nvese of nfoaton at can seve as an estate of covaance at fo vaance paaetes and appopate hypothess testng. 4.3 Restcted Mau Lelhood It s well nown that n a sall saple the au lelhood estaton of vaance paaetes s based. In patcula, f only ntecept s ando (classc egesson) the MLE, ˆ RSS / n s negatvely based. To obtan an 638

4 ASA Secton on Nonpaaetc Statstcs unbased estato RSS /( n ) the estcted MLE should be used. Followng the lne of devaton fo lnea ed odel (Dedeno 4, p. 58) we obtan that the estcted loglelhood dffes fo the usual log-lelhood (7) by te v n T lnxsx ln ln, E whee the n n atces E ae defned as n E,,,...,. To copute the RMLE by Fshe scong we need the fst and second devatves whch ae T t( E E ), T t( E E E ), E (9) whee E E. Then the fst and second devatves of the standad log-lelhood functon, l ae augented by (8) and Fshe scong algoth apples. ˆ 5. Hypothess Testng We can estate the covaance at as ˆ cov( ) ˆ & ( & VLS τ X X ), whee ˆ τ RSS /( n ) and RSS s the esdual su of squaes, the nu value of (4). Fo eaple, we eect H : f ˆ ) ˆ & ( τ ( & VLS > X X ) q α, whe e q α s the ( α) th quntle of the t-dstbuton wth n degees of feedo. Note we use a one-tal hypothess test. Analogously, we can apply UVLS to copute the covaance at usng & ( X P &v X & ). As a wod of cauton, these tests ae not eact because uˆ s not noally dstbuted. The test on andoness of egesson coeffcent s essental. Let be any. We want to test that the th coeffcent s fed (not ando), n othe wods, H :. We use scoe test. We obseve that H would be easonable to eect f l / >, because unde H we have l /, whee the devatve s gven by (8). β 6. Estaton of Rando Coeffcents Afte estates fo fed effects coeffcents and vaance paaetes ae obtaned we ay estate ando coeffcents b. In essence estaton of b s equvalent to estaton of ando effects n a ed effects odel. Also, t should be noted that estaton s undestood not n a classc sense but as a pedcton. Thee ae two equvalent appoaches to estate b assung that vaance paaetes ae nown. Fst, we can estate b as condtonal o posteo ean followng the Bayesan paadg. Indeed, the egesson wth ando coeffcents ay be vewed as condtonal odel fo y b. We ay evese ths to obtan the posteo dstbuton b, patculaly the posteo ean, E b y ). y ( To obtan E( b y ) we obseve that ( y, b ) have a (+) ultvaate noal dstbuton wth ean and covaance at β z β β µ., Ω β Usng standad foula of ultvaate dstbuton we obtan v b E( b y ) β + ( y ) β z β + e () z Second, we can use penalzed least squaes (Dedeno 4, p. 5): 639

5 ASA Secton on Nonpaaetc Statstcs ( y v b ) z + ( b β ) n Dffeentatng wth espect b we obtan e ( ) b β, z () whch gves estate () fo b. It s woth to note that two equal ways of estatng of coeffcents () and () show once agan the elatvty of ntensve dsputes between fequentsts and Bayesans, snce two appoaches vey often gve ust the sae esults dependng on of nzaton functonal, see detals n (Dedeno 4). 6. Monte Calo Sulaton A sulaton was pefoed to copae geneated yelds and those estated by the algoth. Each data set had the sae sze (4 data ponts), wth one o thee factos, and was geneated by a foula (), whee X was taen as a unfoly dstbuted vaable and then ept fed dung epeents. Yelds wee estated by () wth vaance estaton by (5). The effects of fve paaetes wee analyzed.. Eo was noally dstbuted ando vaable wth dffeent vaance and zeo ean. Eo s level was contolled as a ato of ts vaance to the standad devaton of Y wthout an eo. It too levels, %, %, and 4%.. Coelaton between factos was set on zeo and hgh (.6-.7) levels. 3. Coelaton between facto and ts yelds was ethe zeo o sgnfcant (about.5-.6) fo one facto and zeo fo two othes. 4. Geneated yelds (G) level of vaaton was set on low (5%), edu (%-5%), and hgh (%) levels fo all factos. 5. Dffeent yelds vaatons wee obtaned by eepng fst facto s yelds wth hgh vaaton, whle othes ae wth edu. Fo each paaetes cobnaton ando data sets wee geneated, whee fed values of factos wee assgned to ando values of yelds. If vaance estaton n (5) becoes negatve, then the yelds wee not calculated, what gave about 6-75 yelds sets fo each settng. Pat of the esults fo thee factos s pesented n Table n a fo of aveage values fo all uns. The qualty of the yelds appoaton was easued by two statstcs: oot ean squae eo (RMSE, dffeence between actual and estated yeld values) was calculated as a ato of RMSE and geneated yelds ean; coelaton between geneated and estated yelds. The qualty of the ente odel s descbed by detenaton coeffcent R. The an fndngs ae suazed below.. Detenaton afte yeld analyss eveywhee s hghe than afte egesson, whch stesses the fact that the low ognal coelaton ay hde the actual pesence of dependence. Fo eaple, at ow 3, whee no eo s added to Y, the egesson shows a wea dependence (4%), wheeas yeld analyss gves a 98% detenaton. It coesponds wth a fact that outcoe, ndeed, depends entely on those factos weghted by espectve yelds. Eaples le ths llustate the possblty of two dffeent scenaos n ealty, what usually ae nsepaable by statstcal ethods: low detenaton ay be eplaned by hgh levels of ando nose ( envonental easuable nfluence ) o by hgh vayng yelds. Dependng on the odel, ntepetaton of the esults could be copletely dffeent (ows 3,8).. Coelaton between geneated and deved yelds has odeate values, loweng fo.5 to.3 whle eo ses fo zeo to 4%, egadless on yelds vaaton (ows -3). It s had to epect a uch hghe coelaton, eepng n nd the coplety of the poble (fo one facto odel t anges, howeve, fo.7 to.9). But a fact of postve coelaton s vey potant and tells one that soe poveent vs. egesson estaton s always guaanteed. 3. Yelds vaaton does not pactcally affect coelaton between geneated and estated yelds only f all factos have yelds wth the sae vaaton level. It s not tue when equalty does not hold: a facto wth hghest vaaton s ecoveed uch bette copae.6 vs n ows 4,9,4, egadless on level of eo n data (shadowed ows). Its woth to note that n geneal a ecoveng of yelds wth hgh vaaton s oe potant than wth low one, snce the latte wll be captued by usual egesson and does not sgnfcantly affect a odel qualty. 64

6 ASA Secton on Nonpaaetc Statstcs Table. Results of sulaton,, uns fo each settng Coelaton between Geneated geneated G and Factofactofacto Yelds- yelds RMSE, % estated b yelds Yelds vaaton, Regesson R^, Eo coelaton coelaton % G G G 3 G G G 3 R^, % % No No No No No No No No No No No No % No No % No No % No No % Yes No % Yes No No Yes No No Yes No No No Yes % No Yes % No Yes (*) Shadow eans that hgh vaaton was set only fo the fst facto 4. Whle leavng coelatons alost untouched, yelds vaaton defntely deceases the level of appoaton: RMSE s ased fo 4% to aound 8% wth se of vaaton (ows -3), and goes even to % fo the fst facto wth hgh vaaton, whle coelaton of ts yelds wth eal ones s hgh (ow 4). I.e., the hghe coelaton, the lowe appoaton, those two qualty statstcs wo typcally n opposte dectons. Indectly, t s eflected n a fact, that vaatons of yeld estates ae oughly n tes salle than vaaton of eal yelds (not shown n a table). All that eans, that yelds ae bette to use as dectonal ange values, yet dscoveng new aspects of a odel. 5. The ows -4 epesent the esults fo yelds, coelated wth facto values,.e. ay eflect dffeent pattens n data, le dynac tends, nonlneaty, and so on. As seen, the level of ecovey s alost the sae as n stuaton wthout coelatons, as n ows 6,7, 9 and ale. It eans, that those hdden pattens n data ay be evealed n an autoatc way, wthout specal hypotheszng. Indeed, n eal data we found seveal tes, that yelds ae coelated wth a facto, what adds a lot to the pocess undestandng. Fo eaple, t was dscoveed that bg volue of advetsng typcally postvely coelated wth ts outcoes pe unt (Mandel and Hause 5,). The shot suay of dffeent consdeed factos could be found n Table, whch shows the coelatons between aveage values of those factos and aveage values of two dependant vaables. The coelatons ae obtaned fo the table, analogous to Table, whee nstead of No and Yes ae the eal aveage values of sulaton paaetes. Of couse, t s ust a ough ustfcaton of the pocess, but t stll gves suffcent undestandng of what s gong on. The utual coelatons between those fve factos ae qute low, whch allowed calculatng the egesson. The detenaton coeffcents ae povded n the last ow of the Table. As one ay see, the bggest negatve pact to coelaton between geneated and estated yelds s coelaton between factos, the bggest postve effect elatve vaaton (see coent 3 above). The fst fact s an ndect eflecton of the ultcolneaty poble unfotunately, yeld analyss does not povde a eedy fo that by tself (though t helps a lot, because t shfts the poble fo coelaton of factos to coelaton of yelds and othe effects). The second one gves a hnt, that f thee ae soe pelnay deas about the 64

7 ASA Secton on Nonpaaetc Statstcs vaance of yelds, t helps to estate a elablty of yelds obtaned the best ones would be those wth hgh elatve vaance. But snce estates of vaance ae aleady obtaned (see secton 4), t could be used to answe that queston dectly. Table. Coelatons of factos wth chaactestcs of qualty of yelds appoaton Coelaton between geneated and estated Factos yelds RMSE Eo (.3) (.4) Coelaton between factos (.74).7 Yelds vaaton.4.9 Coelaton between facto and yelds.3 (.8) Relatve hgh vaaton of facto's yelds Regesson detenaton,% 9% 9% Chaactestcally, the elatve yelds vaance spols RMSE wth the sae stengths as t poves the coelaton. It eans, that the optal soluton s possble only fo oe o less equally vaable yelds. The level of RMSE s stongly affected by yeld absolute vaaton (coelaton.9), n shap contast to that fo coelaton (.4). Togethe wth the pevous concluson, ths fact allows to estate the level of elablty of yelds estaton afte calculaton of yelds vaances. In geneal, lsted paaetes of data geneaton eplan oe than 9% of qualty statstcs vaaton. Snce thee of the ost potant (coelaton between factos, yelds absolute and elatve vaaton) ae to be nown afte calculatons, those epcal fndngs fo sulaton togethe wth dect estaton of ndvdual yelds allow to say how close ae the estates to eal yelds. 7. Pactcal Ipleentatons On ateals of one lage ca anufactue the yeld analyss was pefoed, soe esults of whch ae pesented below. On Fg. the yelds of advetsng ae shown n dynacs ove te (onths); the scales ae conventonal. It shows, that the effcency of an ad s vey dffeent, wth a slght negatve tend. The net step could be, pesuably, to undestand why fo eaple, to see what eact content of an ad was evey te, what te of the day was t aed, and so on.e., ae a specal odel, whee yelds play a ole of dependent vaable. In Fg. the sae yelds ae shown n elaton wth aount of oney spent fo advetsng evey onth. The sae type of conclusons ay be dawn fo hee. Yelds In spte of slght postve coelaton, n any nstances a lot of oney was spent n van, even wth zeo esult. Togethe wth dynac data le shown on Fg. ths type of analyss povdes anages wth new vey effectve tool to opeate busness. Seveal geneal ecoendatons ay be ade about yelds pleentaton fo statstcal and busness ponts of vew. a) The level of data appoaton n yeld analyss s always hghe than n tadtonal Fgue 3. Advetsng and ts effcency Fgue. Advetsng yelds ove te 4 7 Advetsng egesson-le odels; theefoe t allows one to ceate plausble odels n a uch boade ange of eal lfe stuatons, ncludng those wth low coelatons between the outcoe and factos. b) The pedcton of the outcoe fo new data ponts, whch s staghtfowad n 64

8 ASA Secton on Nonpaaetc Statstcs egesson, s less obvous hee snce yelds ae not constant ove space o te. The best way of dong ths s to assgn new facto values to new yelds, based on foecasted tends, le shown n Fg.. c) If all yelds ae postve t s possble to calculate dect contbutons of a facto to outcoe fo each data pont what s vey potant (a poble () should be odfed espectvely by addng constants). d) Yelds add a new denson to data analyss. Patculaly, one ay do cluste analyss n yeld space and fnd zones of hgh and low yelds fo any vaables, whch wll dffe fo those based on the ognal values of the factos (see Fg. 3). So fa the ente concept of hoogenety was based on clustes n the space of X o Y o X and Y togethe. But yelds conceptually change t by ntoducng vaables wth new eanng. One ay assue that yelds fo the gven data set ae not hoogeneous and efoulate (). On the sae note, new odels usng yelds as dependent vaables ay be bult. e) Yeld analyss ay be used fo dynac data. Ths s a specal topc, assocated wth concept of polonged effcency, whch s patally coveed n (Mandel and Hause, 5,,). 8. Concluson Fo the fst te an eact estatons of the ando coeffcents n egesson odel ae obtaned by seveal ethods and ts elaton wth yeld analyss concept s shown. A pecson and elablty of ando coeffcents estates ae checed though the sulaton epeent wth cobnatons of ost nfluental paaetes. Recoendatons about elatve potance of those factos ae gven; patculaly, t s found that level of yelds ecovey stongly depends on yelds vaaton and factos coelaton. The potance of yeld analyss was deonstated on a level of pactcal pleentaton. The potance of yeld analyss as a genec statstcal tool was ephaszed. It allows ang such new thngs as to detene ndvdual contbutons of factos, dffeentate zones of hgh and low nfluence of all factos, ae clusteng n new space, ceate new type of dynac odels and oe. Two facts ae yeld analyss a unvesal tool fo any pactcal applcatons. Fst, the odel wth ando coeffcents by ts natue s uch oe natual than odel only wth fed effects. Second, a data set fo classcal ed odel should contan seveal subsets of clusteed data, whch s not always avalable, wheeas fo yeld analyss the stuctue of data set s absolutely typcal (odnay tables obect vaables o te-vaables ),.e. t ay be used wheeve egesson and othe tadtonal ethods ay. It allows hopng fo wde use of the poposed appoach. Howeve, not all theoetcal popetes of yelds ae nvestgated yet, and advanced algoths wth bette yelds estates ae to coe. Refeences Dedeno, E. (4). Med Models: Theoy and Applcatons. Hoboen, NJ: Wley. Hldeth, C. and Houc, J.P. (968). Soe estatos fo lnea odel wth ando coeffcents. Jounal of the Aecan Statstcal Assocaton 63, Hube, P.J. (98). Robust Statstcs. New Yo: Wley. Ra, B. (975). Lnea egesson wth ando coeffcents: the fnte saple and convegence popetes. Jounal of the Aecan Statstcal Assocaton 7, Sway, P.A.V.B. (97). Statstcal Infeence n Rando Coeffcent Regesson Models. New Yo: Spnge-Velag. Zaan A. (), Mau lelhood estates fo the Hldeth Houc ando coeffcents odel, Econoetcs Jounal, volue 5, pp. 6. Mandel I. (3) Multcollneaty poble n aetng studes. Jont Statstcal Meetng. Abstacts. Secton Statstcs n Maetng. San Fancsco, p.4 Mandel I., Hause D. (5,) Obect Vayng Coeffcents n Stochastc Envonents: Yeld Analyss to Model Maetng Effcency. The Intenatonal Syposu on Stochastc Models n Relablty, Safety, Secuty and Logstcs, Bee Sheva, Isael, pp Mandel I., Hause D. (5,) Yeld Analyss and Retun on Investent Estaton. Jont Statstcal Meetng Poceedngs, Mnneapols, ths ssue. 643