Varianc Componnts (VARCOP) Notation Th Varianc Componnts procdur provids stimats for variancs of random ffcts undr a gnral linar modl framwor. Four typs of stimation mthods ar availabl in this procdur. Th following notation is usd throughout this chaptr. Unlss othrwis statd, all vctors ar column vctors and all quantitis ar nown. n Numbr of obsrvations, n Numbr of random ffcts, 0 m 0 Numbr of paramtrs in th fixd ffcts, m 0 0 m i Numbr of paramtrs in th ith random ffct, m i 0, i, K, m Total numbr of paramtrs, m m 0 + m + L + m i γ i y Unnown varianc of th ith random ffct, i 0, i, K, Unnown varianc of th rsidual trm, sam as +, > 0 i i Unnown varianc ratio of th ith random ffct, γ, γ i i, K,, and γ + Th lngth n vctor of obsrvations Th lngth n vctor of rsiduals X i Th n m i dsign matrix, i 0,, K, β 0 Th lngth m 0 vctor of paramtrs of th fixd ffcts β i Th lngth m i vctor of paramtrs of th ith random ffct, i, K, 0, Unlss othrwis statd, a p p idntity matrix is dnotd as I p, a p q zro matrix is dnotd as 0 p q, and a zro vctor of lngth p is dnotd as 0 p.
Varianc Componnts (VARCOP) Wights For th sa of clarity and simplicity, th algorithms dscribd in this chaptr assum unit frquncy wight and unit rgrssion wight for all cass. Wights can b applid as dscribd in th following two sctions. Frquncy Wight Th WIGHT command spcifis frquncy wights. Cass with nonpositiv frquncy ar xcludd from all calculations in th procdur. Non-intgral frquncy wight is roundd to th narst intgr. Th total sampl siz is qual to th sum of positiv roundd frquncy wights. Rgrssion Wight odl Th RGWGT subcommand spcifis rgrssion wights. Suppos th lth 0l, K, n5 cas has a rgrssion wight w i > 0 (cass with nonpositiv rgrssion wights ar xcludd from all calculations in th procdur). Lt W diag w, K, w n 6 b th n n diagonal wight matrix. Thn th VARCOP procdur will prform all calculations as if y is physically transformd to W y and X i to W Xi, i 0,, K, ; and thn th prtinnt algorithm is applid to th transformd data. Th mixd modl is rprsntd, following Rao (973), as y X0β0 + X i β i + i
Varianc Componnts (VARCOP) 3 Th random vctors β, K, β and ar assumd to b jointly indpndnt. orovr, th random vctor β i is distributd as N m i40, i I m i9 for i, K, and th rsidual vctor is distributd as N n 4 0, I n 9. It follows from ths assumptions that y is distributd as N n X0β 0, V + V γ ixix i + In γ ivi i i 4 9 whr whr Vi XiXi, i, K,, and Vi In. inimum Norm Quadratic Unbiasd stimat (INQU) Givn th initial guss or th prior valus γ i αi α i 06, i, K, +, th INQU of ar obtaind as a solution of th linar systm of quations: S q B is a 0 5 0 5 symmtric matrix, q q i whr S s ij + + and 4, K, + 9. Dfin 4 9 R V V X 0 X 0 V X 0 X 0 V ;@ is a 0 +5-vctor, Th lmnts of S and q ar sij % & K ' K 3 8 SSQ XRX i j i, K,, j, K, SSQ XR i 6 i, K,, j + SSQ 3RX j8 i +, j, K, SSQ R i +, j + 0 5
4 Varianc Componnts (VARCOP) and % i i qi & ' XRy6, K, SSQ0Ry5 i + whr SSQ(A) is th sum of squars of all lmnts of a matrix A. INQU(0) Th prior valus ar α i 0, i, K,, and α +. Undr this st of prior valus, V I n and R In X0X 0X06 X 0. Sinc this R is an idmpotnt matrix, som of th lmnts of S and q can b simplifid to si, + trac XRX i i i, K, ; s+, j trac XRX j j j, K, ; s+, + n ran X0 q+ yry 6 3 8 6 Using th algorithm by Goodnight (978), th lmnts of S and q ar obtaind without xplicitly computing R. Th stps ar dscribd as follows: Stp. Form th symmtric matrix: X 0X0 X 0X L X 0X X 0y XX 0 XX L XX Xy XX 0 XX L XX Xy yx 0 yx L yx yy
Varianc Componnts (VARCOP) 5 Stp. Swp th abov matrix by pivoting on ach diagonal of XX 0 0. This producs th following matrix: G GX 0X L GX0 X GX 0y XXG 0 XRX L XRX XRy XXG 0 XRX L XRX XRy yx 0G yrx L yrx yry whr G X 0X0 is obtaind as th numbr of nonzro pivots found. 6. In th procss of computing th abov matrix, th ran of X 0 Stp 3. Form S and q. Th INQU(0) of ar $ S q. INQU() Th prior valus ar α i, i, K, +. Undr this st of prior valus, + V Xi Xi. Using Gisbrcht (983), th matrix S and th vctor q ar obtaind i through an itrativ procdur. Th stps ar dscribd as follows: Stp. Construct th augmntd matrix A X0 X L X y. Thn comput th 0m+ 5 0m+ 5 matrix T+ A A 0 5. + Stp. Dfin H05 l Xi Xi, and T05 l A H05 l A i l T05 l using th W Transform givn in Goodnight and Hmmrl (979). Th updating formula is 4 9 05 l 0l+ 5 0l+ 5 l ml l 0l+ 5 l l 0l+ 5 T T A H X I + X H X X H A, l, K,. Updat T0 l+ 5 to
6 Varianc Componnts (VARCOP) 05 05 Stp 3. Onc T A H A A V A is obtaind, apply th Swp opration to th diagonal lmnts of uppr lft m0 m0 submatrix of T05. Th rsulting matrix will contain th quadratic form yry, th vctors yrx j, j, K,, and th matrics XRX i j, i, K,, j, K,. Stp 4. Comput th lmnts of S and q. Sinc RVR R, thn SSQ3RX j8 tr3x jrx j8 SSQ 3X jrxi8 j, K, i SSQ0R5 n ranx06 trxi RXi6 SSQ3RXj8 i j 0 5 3 8 SSQ Ry y Ry SSQ y RX j j Th INQU() of ar $ S q. aximum Lilihood stimat (L) Th maximum lilihood stimats ar obtaind using th algorithm by Jnnrich and Sampson (976). Th algorithm is an itrativ procdur that combins Nwton-Raphson stps and Fishr scoring stps. Paramtrs Th paramtr vctor is θ # β0 γ γ # whr γ γ # $. #
Varianc Componnts (VARCOP) 7 Lilihood Function Th lilihood function is 05 0 5 0 06 0 06 n L L θ π V xp y X β V y X β. Th log-lilihood function is 0 5 4 9 6 6 n n l log L log π log log V y X. 0β0 V y X0β 0 Th Gradint Vctor X β 0V r, 0 rv XXV i i i i i r tr 4XV X9,, K,, γi n rv r. 4 whr r y X 0 β 0. Th gradint vctor is dl dθ β0 γ
8 Varianc Componnts (VARCOP) Th Hssian atrix 0 β β XV X 0 0 0 i i 0 i γ β rv XXV X, K,, i 0 i j 4 9 tr XV i XXV j j Xi rv XXV i j i i XXV j j r, K, ;, K,, 4 β rv γ 0 X 0 rv XjX jv r j, K, 4 j n rv 4 6 r Th Hssian matrix is d l dθθ d β0 β 0 β0 γ β0 γ β0 γ γ γ β0 γ
Varianc Componnts (VARCOP) 9 whr γ β 0 γ β 0 γ β 0, γ γ O L L γ γ and γ γ γ Th Fishr Information atrix As 0r5 0n and rv r n XV 0 0 X 0 β β0, th xpctd scond drivativs ar m i 0 γi β0 0, K, i j j i i j tr 4XV XXV X9, K,,, K, γi γ j
0 Varianc Componnts (VARCOP) m0 β0 0 j j j tr 4XV X9, K,, γ j n 4 Th Fishr Information matrix is d l dθθ d whr and γ γ β0 β0 0 m m0 m0 6 0 m0 0 0 m m m m 0 0 0 γ γ γ γ 6 L O γ γ L γ γ
Varianc Componnts (VARCOP) γ γ γ Itration Procdur Initial Valus 6. 0 5 random ffct, 6. Thn assign th varianc of th m i lmnts of β $ i 6 to th stimat $ i if mi ; othrwis $ i 0. Fixd ffct Paramtrs: β $ 0 XX 0 0 Xy 0 Random ffct Varianc Componnts: For th ith i, K, comput β $ i XX i i Xy i using divisor m i Rsidual Varianc: $ rr n whr X X 0 X L X and 8 r y 0X X5 X y. If $ 0 but thn rst $ 0 so that th itration can continu. Th varianc ratios ar thn computd as $ γ $ i i $, i, K,. Following th sam mthod in which th rsidual varianc is initializd, $ > 0 for. Updating At th sth itration 0s 0,,K5, th paramtr vctor is updatd as 0 5 05 05 θ$ θ$ θ$ s+ s + ρ s whr θ $ 05 s is th valu of incrmnt θ valuatd at θ θ $ s $ $ 0 5 0 5 siz such that l θ s+ l θ s 05, and ρ > 0 is a stp 4 9 4 9. Th incrmnt vctor dpnds on th choic of stp typ Nwton-Raphson vrsus Fishr scoring. Th stp siz is dtrmind by th stp-halving tchniqu with ρ initially and a maximum of 0 halvings.
Varianc Componnts (VARCOP) Choic of Stp Following Jnnrich and Sampson (976), th first itration is always th Fishr scoring stp bcaus it is mor robust to poor initial valus. For subsqunt itration th Nwton-Raphson stp is usd if:. Th Hssian matrix is nonngativ dfinit, and. Th incrmnt in th log-lilihood function of stp is lss than or qual to on. Othrwis th Fishr scoring stp is usd. Th incrmnt vctor for ach typ of stp is: Nwton-Raphson Stp: θ Fishr Scoring Stp: θ d l θθ d d θθ d l d d dl. dθ dl. dθ Convrgnc Critria Givn th convrgnc critrion ε > 0, th itration is considrd convrgd whn th following critria ar satisfid:. l θ$ l θ$ ε max, l θ$, and 4 0s+ s < 59 4 059 4 05 s 9 θ$ θ$ max, θ$ 4 s+ s 9 < 4 s 9 whr a is th sum of absolut valus 0 5 05 05. ρ ε of lmnts of th vctor a. Ngativ Varianc stimats Ngativ varianc stimats can occur at th nd of an itration. An ad hoc mthod is to st thos stimats to zro bfor th nxt itration.
Varianc Componnts (VARCOP) 3 Covarianc atrix Lt $ θ b th vctor of maximum lilihood stimats. Thir covarianc matrix is givn by 49 cov θ $ d l dθθ d $ θ θ Lt β ψ 0 b th original paramtrs. Thir maximum lilihood stimats ar givn by ψ$ β$ 0 $ $ γ $ $ γ $ and thir covarianc matrix is stimatd by 7 49 cov ψ$ Jcov θ $ J
4 Varianc Componnts (VARCOP) whr J Im 0m 0m 0 m I 0 γ 0 0 0 0 0 6 6 Jacobian matrix of transforming θ to ψ. which is th m0 + + m0 + + Rstrictd aximum Lilihood stimat (RL) Th rstrictd maximum lilihood mthod finds a linar transformation on y such that th rsulting vctor dos not involv th fixd ffct paramtr vctor β 0 rgardlss of thir valus. It has bn shown that ths linar combinations ar th rsiduals obtaind aftr a linar rgrssion on th fixd ffcts. Suppos r is th ran of X 0 ; thn thr ar at most n r linarly indpndnt combinations. Lt K b an n 0n r5 matrix whos columns ar ths linarly indpndnt combinations. Thn th proprtis of K ar (Sarl t al., 99, Chaptr 6): KX 0 0 n r m0 K T 0 5 whr T is a 0n r5 n matrix with linarly indpndnt rows and 6 In X0 X 0X0 X 0 It can b shown that RL stimation is invariant to K (Sarl t al., 99, Chaptr 6); thus, w can choos K such that KK I n r to simplify calculations. It follows that th distribution of Ky is N n r 40, K VK9. Paramtrs Th paramtr vctor is θ $ # γ γ whr γ γ # $. #
Varianc Componnts (VARCOP) 5 Lilihood Function Th lilihood function of Kyis n r 05 0 50 5 4 0 5 9 L L θ π KVK xp ykkvk Ky. It can b shown (Sarl t al., 99) that 4 9 0 5 R V V X 0 X 0 V X 0 X 0 V K K VK K Thus, th log-lilihood function is 0 5 4 9 KVK yry l n r n r L log log log log π. Th Gradint Vctor γ yrxxry i i tr XRX i i i, K, i 0 5 n r yry 4 6 Th gradint vctor is dl dθ γ
6 Varianc Componnts (VARCOP) Th Hssian atrix γ yrxxrxxry i i j j + tr XRXXRX i j j i i,, ; j,, i j yrxxry j j j, K, 4 j n r yry + 6 4 3 8 K K Th Hssian matrix is d l dθθ d γ γ γ γ whr γ γ O L L γ γ and γ γ γ Th Fishr Information atrix Sinc KX 0n r m 0 5 and trac RV 0 0 0 5 n 3 8 K K r, th xpctd scond drivativs ar i j j i i j tr XRX X RX,,,,, γi γ j
Varianc Componnts (VARCOP) 7 j j j tr XRX,, γ j n r 4 3 8 K Th Fishr Information matrix is d l dθθ d whr and γ γ γ γ γ L O γ γ L γ γ γ γ
8 Varianc Componnts (VARCOP) Itration Procdur Initial Valus 0 5 random ffct, Random ffct Varianc Componnts: For th ith i, K, comput β $ i XX i i 6 Xy i. Thn assign th varianc of th m i lmnts of β $ i using divisor m i 6 to th stimat $ i if mi, othrwis $ i 0. Rsidual Varianc: $ rr0 n r5 whr X X0 X L X and 8 r y 0X X5 X y. If $ 0 but thn rst $ 0 so that th itration can continu. Th varianc ratios ar thn computd as $ γ $ i i $, i, K,. Following th way th rsidual varianc is initializd, $ > 0 for. Updating At th sth itration 0s 0,,K5, th paramtr vctor is updatd as 0 5 05 05 θ$ θ$ θ$ s+ s + ρ s whr θ $ 05 s is th valu of incrmnt θ valuatd at θ θ $ s $ $ 0 5 0 5 siz such that l θ s+ l θ s 05, and ρ > 0 is a stp 4 9 4 9. Th incrmnt vctor dpnds on th choic of stp typ Nwton-Raphson vrsus Fishr scoring. Th stp siz is dtrmind by th stp-halving tchniqu with ρ initially and a maximum of 0 halvings. Choic of Stp Following Jnnrich and Sampson (976), th first itration is always th Fishr scoring stp bcaus it is mor robust to poor initial valus. For subsqunt itrations, th Nwton-Raphson stp is usd if:. Th Hssian matrix is non-ngativ dfinit, and. Th incrmnt in th log-lilihood function of prvious stp is lss than or qual to on.
Varianc Componnts (VARCOP) 9 Othrwis, th Fishr scoring stp is usd instad. Th incrmnt vctor for ach typ of stp is: Nwton-Raphson Stp: θ Fishr Scoring Stp: θ d l θθ d d θθ d l d d dl. dθ dl. dθ Convrgnc Critria Givn th convrgnc critrion ε > 0, th itration is considrd convrgd whn th following critria ar satisfid:. l θ$ l θ$ ε max, l θ$, and 4 0s+ s < 59 4 059 4 05 s 9 θ$ θ$ max, θ$ 4 s+ s 9 < 4 s 9 whr a is th sum of absolut valus 0 5 05 05. ρ ε of lmnts of th vctor a. Ngativ Varianc stimats Ngativ varianc stimats can occur at th nd of an itration. An ad hoc mthod is to st thos stimats to zro bfor th nxt itration. Covarianc atrix Lt $ θ b th vctor of maximum lilihood stimats. Thir covarianc matrix is givn by 49 cov θ $ d l dθθ d $ θ θ
0 Varianc Componnts (VARCOP) Lt ψ b th original paramtrs. Thir maximum lilihood stimats ar givn by ψ $ $ $ γ $ $ γ $ and thir covarianc matrix is stimatd by 7 49 cov ψ$ Jcov θ $ J whr J I γ 0 0 5 0 5 Jacobian matrix of transforming θ to ψ. which is th + + ANOVA stimat Th ANOVA varianc componnt stimats ar obtaind by quating th xpctd man squars of th random ffcts to thir obsrvd man squars. Th VARCOP procdur offrs two typs of sum of squars: Typ I and Typ III (s Appndix for dtails).
Varianc Componnts (VARCOP) Lt ψ b th vctor of varianc componnts. Lt q S S S whr S i, i, K, is th obsrvd man squars of th ith random ffct, and S is th rsidual man squars. Lt S s s s + $ # 0 5 0 5 matrix whos rows ar cofficints for th xpctd man b a + + squars. For xampl, th xpctd man squars of th ith random ffct is s i ψ. Algorithms for computing th xpctd man squars can b found in th sction Univariat ixd odl in th chaptr GL Univariat and ultivariat. Th ANOVA varianc componnt stimats ar thn obtaind by solving th systm of linar quations: Sψ q
Varianc Componnts (VARCOP) Rfrncs Gisbrcht, F. G. 983. An fficint procdur for computing INQU of varianc componnts and gnralizd last squars stimats of fixd ffcts. Communications in Statistics, Sris A: Thory and thod, : 69 77. Goodnight, J. H. 978. Computing IVQU0 stimats of Varianc Componnts. SAS Tchnical Rport R-05, Cary, N.C.: SAS Institut. Goodnight, J. H., and Hmmrl, W. J. 979. A simplifid algorithm for th W transformation in varianc componnt stimation. Tchnomtrics, : 65 67. Jnnrich, R. I., and Sampson, P. F. 976. Nwton-Raphson and rlatd algorithms for maximum lilihood varianc componnt stimation. Tchnomtrics, 8: 7. Rao, C. R. 973. Linar statistical infrnc and its applications, nd d. Nw Yor: John Wily & Sons, Inc. Sarl, S. R., Caslla, G., and cculloch, C.. 99. Varianc componnts. Nw Yor: John Wily & Sons, Inc.