Pa. J. Statst. 015 Vol. 31(5) 447-460 VARIABLE SELECION IN JOIN MEANS AND VARIANCE MODELS OF HE PAREO DISRIBUION Yg Dog 1 Lx Sog 1 Muhammad Am 13 ad Xyog Sh 4 1 School of Mathmatcal Sccs Dala Uvrsty of chology Dala 11603 P. R. Cha. Emal: lxsog@dlut.du.c Faculty of Scc Dala Natoalts Uvrsty Dala 116600 P. R. Cha. Emal: ygd 66@163.com 3 Nuclar Isttut for Food ad Agrcultur (NIFA) 446 Pshawar Pasta. Emal: amaju@gmal.com 4 roops 68048 h Chs Popl s Lbrato Army Baoj 71013 P. R. Cha. Emal: sxy74@163.com ABSRAC Parto Dstrbuto s a powrful law of probablty dstrbuto that accords wth cty populatos actuaral gophyscal sctfc ad othrs. Varabl slcto s vtal th modlg of statstcs ad also for Parto dstrbuto. It s a gral udrstadg that most of th xstg varabl slcto mthods ar cofd to ma xplaato varabls oly. I ths ss varous jot ma ad varac modls ar also vstgatd to rdf Parto Dstrbuto Modl. A ufd pald llhood mthod whch ca smultaously slct sgfcat varabls th ma ad varac modls s proposd. h cosstcy ad th oracl proprty of th rgulard stmators s also stablshd wth th hlp of apt mod of slctg tug paramtrs. Ft sampl prformac of th proposd varabl slcto procdur s assssd usg dffrt smulato studs. KEYWORDS Baysa formato crtro (BIC); Jot ma ad varac modls; Oracl proprty; Varabl slcto 1. INRODUCION h Parto Dstrbuto amd aftr th Itala coomst Vlfrdo Parto was proposd frst as a modl for th dstrbuto of cty populatos wth a gv ara. O of ts modr uss s to utl as modl for th dstrbuto of coms. Lomax (1954) mployd t th aalyss of busss falur data whl Balma ad D Haa (1974) prstd that t arss as a lmt dstrbuto of lgrg lftm at grat ag. Bryso (1974) dorsd ts us as a havy tald altratv to th xpotal. O th othr t s also a powrful law of th probablty dstrbuto that cocds wth actuaral gophyscal sctfc ad may othr typs of obsrvabl sgularts. Varabl slcto has th vtal sgfcac statstcal modlg. Usually vstgators troduc a larg umbr of prdctors ordr to rduc possbl modl bass but may cass th umbr of mportat covarats s rlatvly small so t s 015 Pasta Joural of Statstcs 447
448 Varabl slcto jot mas ad varac modls rasoabl to assum a spars modl. hrfor t s a d of varabl slcto to dtfy most mportat varabls that provds mor trprtabl modls wth bttr prdcto powr. Most xstg varabl slcto procdurs ar oly lmtd to slct th ma xplaato varabls. Nvrthlss modlg th varac wll b of drct trst ts ow rght to dtfy th sourc of varablty th obsrvatos may stuatos such as dustral qualty mprovmt xprmts ad coomtrc sctor. hus t s as mportat as that of th ma. I prst ra a colossal cosdrato s mad to jot ma ad varac modl. I ths vw Par (1966) proposd to a log lar modl for th varac paramtr ad practcd a two stag procss to stmat th Gaussa modl. Harvy (1976) dlbratd Maxmum Llhood (ML) stmato of th locato ad scal ffcts ad th succdg llhood rato tst udr th gral codtos. At (1987) projctd th ML stmato for a jot ma ad varac modls ad appld t to th commoly ctd Mtab tr data. Outlrs ar commo to b obsrvabl so thr accommodato s of trst rathr tha dlto. aylor ad Vrbyla (004) proposd jot modlg of locato ad scal paramtrs of th t-dstrbuto. Grally dstrbutos from th famly of grald lar modls ar cosdrd by L ad Nldr (1998) Smyth ad Vrbyla (1999) ad by Wag ad Zhag (009) as wll. All ths cocludd to stmat th ma ad dsprso paramtrs of th dstrbuto udr th doubl grald lar modls. Wu ad L (01) cofrrd th varabl slcto for jot ma ad dsprso modls of th vrs Gaussa dstrbuto. Wu t al. (01) dlvrd to th modl of Box-Cox trasformato about th jot ma ad varac ad do th sam o th sw-ormal dstrbuto th yar 013 as wll. Wu t al. (01) sprd us to formulat th slcto of vtal xplaatory varabls whch s th bacbo of jot ma ad varac modls of th Parto dstrbuto. h proposd modl s quppd wth ths rqurmt. h cosstcy ad th oracl proprty of th rgulard stmators wth th hlp of apt mod of slctg tug paramtrs s stablshd. Ft sampl prformac of th proposd varabl slcto procdur s assssd through smulato studs. Som of ths dvlopmts ar vry clos to th rsarch wor by Wu (014). h cotts of ths artcl ar orgad as follows. Frst th jot ma ad varac modls of th Parto Dstrbuto ar proposd Scto ad th dscuss th varabl slcto mthod for ths modls va th pald llhood fucto. Furthrmor som statstcal proprts of our varabl slcto procdur ar prstd. h tratv algorthm to comput th pald maxmum llhood stmators udr th proposd modls s prstd Scto 3. h smulato studs ar dscussd Scto 4 to llustrat th proposd mthodologs.. VARIABLE SELECION IN JOIN MEAN AND VARIANCE MODELS OF HE PAREO DISRIBUION VIA PENALIZED MAXIMUM LIKELIHOOD.1 Jot Ma ad Varac Modls of th Parto Dstrbuto Cosdr th followg jot ma ad varac modls of th Parto dstrbuto:
Dog Sog Am ad Sh 449 y ~ Parto a d x d E y 1 Var y 1. d 1 (.1) d whr f y d 0 0. 1 d y 1 y y y y s a vctor of dpdt rsposs ad rprsts th sampl s. ad X x1 x x ad 1 q Z 1 ar covarats whr x x xp 1. h may cota som or all of th varabls x ad othr varabls whch ar ot cludd paramtrs 1 p x. 1 p s a p 1 vctor of uow s q 1 vctor of uow paramtrs. I ths papr w procdur s proposd to rmov th ucssary xplaatory varabls from jot ma ad varac modls of th Parto dstrbuto.. Pald Maxmum Llhood May tradtoal varabl slcto mthods ca b cosdrd as a pald llhood to balac th modl bass ad stmato varacs (Fa ad L 001). y x 1. from th jot ma ad Suppos that w hav a radom sampl varac modls of th Parto dstrbuto. Lt L( ) dot th log-llhood fucto. h w hav 1 L( ) l ld 1 l y l 1 1l 1 1 1 l y x l 1 1
450 Varabl slcto jot mas ad varac modls whr whr x x x 1 l x 1 x l y 1 l 1 l x 1 1 1 l y 1 x. Smlar to Fa ad L (001) w df th pald llhood fucto as follows p q j Q ( ) L ( ) P P (.) 1j j1 1 P s a pr-spcfd palty fucto (such as LASSO ad SCAD) wth a rgularato paramtr λ whch ca b chos by a data-drv crtro such as crossvaldato (CV) grald cross-valdato (GCV Fa ad L 001; bshra 1996) ad Bays formato crtro (BIC). I ths papr w cosdr thr palty fuctos: last absolut shrag ad slcto oprator (LASSO) smoothly clppd absolut dvato (SCAD) ad CP (Wag t al. 010; Am t al. 015; Dog t al. 014 hr w usd th combato of SCAD wth Rdg). W usd BIC to choos th tug paramtrs ths papr. I ths study lt 1 s 1 p 1 q wth s p q ; w oft us th followg pald llhood fucto: whr s j1 1j Q ( ) L ( ) P j (.3) ( ) l 1 l L x 1 1 1 1 l y
Dog Sog Am ad Sh 451 ad x. 1j j ad SCAD). P s a pr-spcfd palty fucto (such as LASSO h pald maxmum llhood stmator of dotd by ˆ maxms th fucto Q( ) (.3) xcpt for a costat trm. Wth approprat palty fuctos maxmg Q( ) wth rspct to lads to crta paramtr stmators vashg from th tal modls so that th corrspodg xplaatory varabls ar automatcally rmovd. Hc through maxmg Q( ) w achv th goal of slctg mportat varabls ad obtag th paramtr stmats smultaously..3 hortcal Proprts W cosdr th cosstcy ad asymptotc ormalty of th pald llhood stmator ths subscto. Frstly w troduc som otatos. Lt 0 dot th tru valu of. Furthrmor lt (1) () 0 01 0 0s 0 0 wthout loss (1) of gralty t s assumd that 0 cossts of all o-ro compots of 0 ad that () 0 0. I addto w suppos that th tug paramtrs hav b rarragd wth rspct to th lmts of 0. Lt s 1 b th dmso of (1) 0 a ' '' max P 0 0 0 b P j j 1js max 0 0j 0j 1js o obta th proprty of cosstcy ad asymptotc ormalty w rqur th followg rgularty codtos o our modl. (A): h covarat vctors 1 x x xp ad q ar fxd ad boudd. 1 1 (B): h tru valu 0 s th tror of th paramtr spac. (C): h y 1 ar dpdt our modl. horm.1 (Cosstcy). 1 Assum a Op ( ) b 0 ad 0. s qual to thr 1 or dpdg o whthr 0 j s a compot of 0 or 0 1. Udr codtos (A)-(C) wth probablty tdg to 1 thr xsts a local maxmr ˆ of th pald llhood fucto Q( ) quato (.3) such that
45 Varabl slcto jot mas ad varac modls ˆ O ( ). 0 1 p whr h w cosdr th asymptotc ormalty of ˆ. Lt '' (1) '' (1) 01 s1 A dag P P ' ' (1) (1) (1) (1) 01 sg 01 s sg s d dag P P (1) 0 j s th j-th compot of matrx of by I ( ). horm. (Oracl proprty). Assum that th palty fucto 0 ' lm f lm f P / 0 ad wh 0 (1) 0 1 1 j 1 s ' P satsfs 1 dot th Fshr formato I I covrgs to a ft ad postv dft matrx I 0. Mawhl udr th codtos of horm.1 f 0 ad as th th -cosstt stmator ˆ ˆ (1) ˆ () must satsfy () () (Sparsty) ˆ 0 ; () (Asymptotc ormalty) whr 1 1 (1) ˆ (1) (1) (1) L 0 s 0 1 s1 I I A I A d N I L stads for th covrgc dstrbuto ad I s th s 1 s 1 submatrx of I corrspodg to (1) 0 ad s 1 (1) I s s 1 s 1 dtty matrx. horm.1 Rmar: h horm. stads for th Oracl proprty of th stmator udr th modl of (.1). Proofs of horms.1 ad horm. ar sstally th sam as Fa ad L (001). o sav spac th proofs ar omttd. 3. COMPUAION W mploy a algorthm to obta th llhood stmato jot ma ad varac modls of th Parto dstrbuto ths subscto. W also gv th mthod of how to choos th tug paramtrs.
Dog Sog Am ad Sh 453 3.1 Computato of th Llhood Estmato Jot Ma ad Varac Modls of th Parto Dstrbuto Frstly w fd that th frst two drvatvs of th log-llhood fucto L( ) ar cotuous. For a gv pot 0 th log-llhood fucto ca b approxmatd by L 0 1 L 0 0 0 0 0 L( ) L For th gv 0 as P P P P ca also b locally approxmatd by a quadratc fucto ' 1 for 0. 0 0 0 0 hrfor th pald llhood fucto (.3) ca b locally approxmatd by L 0 1 L 0 0 0 0 0 0 Q( ) L whr P 0p P P 0q P 11 01 1p 1 01 q 0 dag 01 0 p 01 0q 1 s 1 p 1 q ad 0 01 0 0s 01 0 0 p 01 0 0q. Accordgly th quadratc maxmato problm for Q( ) lads to a soluto tractd by 1 L0 L0 1 0 0 0 0 Scodly udr th modl of Parto dstrbuto th log-llhood fucto L( ) ca b wrtt as L( ) L( ) l 1 1 1 l x 1 x 1 whr. hrfor th rsultg fuctos ar l y
454 Varabl slcto jot mas ad varac modls L( ) U( ) U1 ( ) U ( ) whr 1 1 x L U1( ) x l x x x 1 1 1 y 1 x 1 1 x 1 1 1 1 1 l l 1 1 x y x L 1 U( ) 1 1 1 1 ad w dot L L L( ) H ( ) whr L L 1 1 x L 1 1 3 1 x l x x x 1 1 3 y 1 x 1 1 1 1 x x x x 1 1 1 4 3 x x x 4 6 4 5 1 1 4 1 x 1 xx 1 x x x
Dog Sog Am ad Sh 455 whr L 1 1 1 x x x 3 3 1 1 l 3 1 y 1 1 1 1 1 4 1 1 x x 1 1 x x x x 1 x 1 1 1 x l 1 x 1 1 1 1 1 x 4x 3 x 1 3 x 1 1 L 1 1 L 1 1 4 1 x 3 1 3 l y x 1 1 x 1 l l 1 y x 1 3 1 1 x 1 1 l 1 1 x. 1 l x y x 1 3 x Fally th followg algorthm summars th computato of pald maxmum llhood stmators of th paramtrs modl (.1).
456 Varabl slcto jot mas ad varac modls Algorthm: Stp 1. a th ordary maxmum llhood stmators (wthout palty) ˆ MLE ad ˆ MLE. (0) ˆ MLE 0 0 0 Stp. Gv th currt valus (0) ˆ MLE of as thr tal valus that s. l l l l l 1 1 H U l l l l l l l Stp 3. Rpat stp utl crta covrgc crtra ar satsfd. updat 3. Slcto of th ug Paramtrs Implmtg th mthods dscrbd abov w d to stmat th thrshold paramtrs. Wag t al. (007) foud that th BIC-typ crtro s cosstt modl slcto ad vrfd that th pald stmator wth th tug paramtr slctd ca dtfy th tru modl cossttly. Followg ths da th BIC s usd to BIC L ˆ d l( ) whr choos th tug paramtrs. h formula of BIC s d s th umbr of oro coffcts of ˆ ad L L ˆ ˆ ˆ ˆ ˆ l 1 1 ˆ ˆ 1 l ˆ 1 l 1 ˆ x y 1 ˆ 1 1 ˆ x whr ˆ ad ˆ ar th pald maxmum llhood stmators. Fa ad L (001) umrcally showd that a 3.7 mms th Baysa rs ad rcommdd ts us practc. hus w st a 3.7. It s xpctd that th choc of 1 j ad should satsfy th tug paramtr for ro coffct s largr tha for o-ro coffct. hus w ca smultaously ubasdly stmat a largr tha that for o-ro coffct ad shr th smallr coffct towards ro. Hc 0 practc w suggst tag ˆ 0 0 0 whr ad ˆ ar th 1 j j ˆ tal stmators of j ad j 1 p; 1 q rspctvly by usg upald maxmum llhood stmators of ad. h tug paramtr ca b obtad as ˆλ = arg m BIC( ). ˆ j
Dog Sog Am ad Sh 457 4. SIMULAION SUDY I ths scto w coduct som Mot Carlo smulatos wth jot ma ad varac modls of th Parto dstrbuto to valuat th ft sampl prformac of th proposd mthodologs. W smulat data from modl (.1) y ~ Parto d x d E y 1 Var y 1. d 1 h coffcts ad ca b computd th formula of 0 (1.501000000). o prform ths smulato w ta 0 (011001.5000000000) 0 (1.501000000). All of th smulato rsults ar basd o 1000 dpdt rpttos. h avrag umbr of th stmatd ro coffcts for paramtrs modl (.1) wth 1000 smulato rus s rportd abl 1. I abl 1 "C ˆ ad C ˆ " gvs th avrag umbr of ro coffcts th colum labld corrctly st to ro ad th colum ˆ ad I ˆ ad " I " gvs th avrag umbr of oro coffcts corrctly st to ro. Furthrmor th colum labld GMSE" gvs th grald ma squar rror of ˆ ad ˆ. Smlar to what L ad Lag (008) Zhao ad Xu (010) had do th prformac of stmators ˆ assssd by usg th grald ma squar rror (GMSE) dfd as GMSE GMSE ˆ ˆ ˆ 0 ˆ ˆ XX 0 ˆ ˆ ˆ ˆ ˆ 0 0 ad ˆ wll b h sampl s th smulatos s 00. As sam as Huag Ma ad Zhag (008) w cosdr two cass whch ar xhbtd th followg two xampls. Exampl 4.1 (Gral) h covarats x ar th multvarat ormal dstrbutos wth ma 0 ad covarac btw th -th ad j -th lmts bg Ad th covarats hav th sam dstrbuto wth j r wth r 0.1 ad r 0.9. x. y s gratd accordg to modl (.1). h rsults of Ma ad Varac Modl ar show abl 1 ad abl rspctvly.
458 Varabl slcto jot mas ad varac modls abl 1 h Smulato Rsult for Ma Modl Exampl 4.1 r 0.1 r 0.9 Mthod GMSE ˆ C ˆ I ˆ GMSE ˆ C ˆ I ˆ 00 Lasso 0.654 4.870 0.000 0.488 4.630 0.000 SCAD 0.377 5.918 0.000 0.394 6.60 0.000 CP 0.405 5.535 0.000 0.335 6.931 0.000 abl h Smulato Rsult for Varac Modl Exampl 4.1 r 0.1 r 0.9 Mthod GMSE C I GMSE C I ˆ ˆ 00 Lasso 0.549 8.870 0.000 0.744 8.15 0.00 SCAD 0.398 9.905 0.000 0.398 9.913 0.010 ˆ CP 0.40 9.4 0.000 0.375 10.001 0.017 From th abl 1 ad abl t ca b show that th prformac of SCAD s much bttr tha th Lasso ad CP th two modls wh r 0.1. But wh th corrlato s gt hghr th prformac of th CP s a lttr bttr tha th Lasso ad SCAD wh r 0.9. So w ca fd that th CP ad SCAD ar always bttr tha th Lasso th two modls bcaus th Lasso dos ot hav th oracl proprty. Exampl 4. (Group Structur) h covarats x ( 1 ) ar gratd as follows: x ~ N(01) ( 1 6) x x 4 wh 7 10 whr ar..d. N (00.01). h covarats j ( j 1 ) ar gratd as follows: jt ~ N(01)( t 1 9) jt jt 6 t wh t 10 15 whr ar..d. N (00.01). y s gratd accordg to modl (.1). h rsults of Ma ad Varac Modl ar show abl 3 ad abl 4 rspctvly. abl 3 h Smulato Rsult for Ma Modl Exampl 4. Mthod GMSE ˆ C ˆ I ˆ 00 Lasso 1.134 5.70 0.000 SCAD 1.006 6.111 0.000 CP 0.988 6.535 0.000 ˆ ˆ ˆ
Dog Sog Am ad Sh 459 abl 4 h Smulato Rsult for Varac Modl Exampl 4. I Mthod ˆ ˆ ˆ GMSE C 00 Lasso 1.454 8.870 0.00 SCAD 1.377 10.005 0.000 CP 1.371 10.435 0.001 It s obvous that th corrlato s vry hgh ths xampl. So from th abl 3 ad abl 4 th prformac of th CP s a lttl bttr tha th Lasso ad SCAD whl th SCAD s bttr tha Lasso th two modls. h raso s that th Lasso dos ot hav th oracl proprty. 5. CONCLUSION AND DISCUSSION I ths papr a w procdur s proposd whch ca slct ad stmat th sgfcat varabls smultaously jot ma ad varac modls of th Parto dstrbuto. Mawhl th cosstcy ad th oracl proprty of th rgulard stmators s also stablshd wth th approprat mthod of slctg th tug paramtrs. h ft sampl prformac of th proposd modl through smulato studs s assssd. h cocluso volvs th futurstc vws of th proposd modl rsarch flds. It wll b mor bfcal to utl t for ft umbr of paramtrs as ths modl s vald oly for fxd umbr of paramtrs. Now as th owldg s broadg day by day so t s sstal to l w doors of thors to old os. Hc t s dd a d to dvlop som w mthods to obta th varabl slcto th jot ma ad varac modls wth dffrt dstrbuto. REFERENCES 1. At M. (1987). Modllg varac htrogty ormal rgrsso usg GLIM. Appl. Stat. 36 33-339.. Am M. Sog L. horl M.A. ad Wag X. (015). Combd pald quatl rgrsso hgh dmsoal modls. Pa. J. Statst. 31 49-70. 3. Balma A.A. ad D Haa L. (1974). Rsdual lf tm at grat ag. A. Probab. 79-804. 4. Bryso M.C. (1974). Havy tald dstrbutos: Proprts ad tsts. chomtrcs 16 61-68. 5. Dog Y. Sog L. Wag M. ad Xu Y. (014). Combd-pald llhood stmatos wth a dvrgg umbr of paramtrs. Joural of Appld Statstcs 41 174-185. 6. Fa J. ad L R. (001). Varabl slcto va ococav pald llhood ad ts oracl proprts. J. Amr. Statst. Assoc. 96 1348-1360. 7. Harvy A.C. (1976). Estmatg rgrsso modls wth multplcatv htroscdastcty. Ecoomtrca 44 460-465. 8. Huag J. Ma S.C. ad Zhag H. (008). Adaptv Lasso for spars hghdmsoal rgrsso modls. Statst. Sca 18 1603-1618.
460 Varabl slcto jot mas ad varac modls 9. L Y. ad Nldr J.A. (1998). Grald lar modls for th aalyss of qualty mprovmt xprmts. Ca. J. Stat. 6 95-105. 10. L R. ad Lag H. (008). Varabl slcto smparamtrc rgrsso modlg. A. Stat. 36 61-86. 11. Lomax K.S. (1954). Busss falurs. Aothr xampl of th aalyss of falur data. J. Amr. Statst. Assoc. 49 847-85. 1. Par R.E. (1966). Estmato wth htroscdastc rror trms. Ecoomtrca 34 888. 13. Smyth G.K. ad Vrbyla A.P. (1999). Adjustd llhood mthods for modllg dsprso grald lar modls. Evromtrcs 10 696-709. 14. aylor J.. ad Vrbyla A.P. (004). Jot modllg of locato ad scal paramtrs of th t dstrbuto. Stat. Modl 4 91-11. 15. bshra R.J. (1996). Rgrsso shrag ad slcto va th Lasso. J. Roy. Statst. Soc. B. 58 67-88. 16. Wag D.R. ad Zhag Z.Z. (009) Varabl slcto jot grald lar modls. Ch. J. Appl. Probab. Stat. 5 45-56. 17. Wag H. L R. ad sa C.L. (007). O th cosstcy of SCAD tug paramtr slctor. Bomtra 94 553-568. 18. Wag X.M. Par. ad Carrr K.C. (010). Varabl slcto va combd palato for hgh-dmsoal data aalyss. Comput. Stat. Data Aal. 54 30-43. 19. Wu L.C. ad L H.Q. (01). Varabl slcto for jot ma ad dsprso modls of th vrs Gaussa dstrbuto. Mtra 75 795-808. 0. Wu L.C. ad Zhag Z.Z ad Xu D.K. (01). Varabl slcto jot ma ad varac modls of Box-Cox trasformato. Joural of Appld Statstcs 39 543-555. 1. Wu L.C. ad Zhag Z.Z. ad Xu D.K. (013). Varabl slcto jot locato ad scal modls of th sw-ormal dstrbuto. Joural of Statstcal Computato ad Smulato 83 166-178.. Wu L.C. (014). Varabl slcto jot locato ad scal modls of th sw-tormal dstrbuto. Commucatos Statstcs - Smulato ad Computato. 43(3) 615-630. 3. Wu L.C. Zhag Z.Z. a G.L. ad Xu D.K. (014). A robust varabl slcto to t-typ jot grald lar modls va pald t-typ psudo-llhood Commu. Statst. Smul. ad Compu. DOI: 10.1080/03610918.014.901358. 4. Zhao P.X. ad Xu L.G. (010). Varabl slcto for smparamtrc varyg coffct partally lar rrors--varabls modls. J. Multvarat Aal. 101 187-1883.