The Distribution of Eigenvalues of Covariance Matrices of Residuals in Analysis of Variance

Transcription

1 JOURNAL OF RESEARCH of the Natonal Bureau of Standards - B. Mathem atca l Scence s Vol. 74B, No.3, July-September 1970 The Dstrbuton of Egenvalues of Covarance Matrces of Resduals n Analyss of Varance John Mandel Insttute for Materals Research, Natonal Bureau of Standards, Washngton, D.C (Aprl 9, 1970) A r gorous defnton s gven for th e concept of an "n te racton ma trx " (Z) where = 1 to In and = 1 to n, n te rm s of two dempote nt matr ces A" and Bs of rank rand s, respec tvely. It s then shown that the frequency dstrb uton of the egenvalues of (Z)(Z), depends only on rand s. App l ca tons are gven to matr ces of resdu\ls arsng from two-way data, e the r by removng rowand/or colu mn-means, or by applyng any number of sweeps of the "vacuum cleaner." Th e th eore ms are mportant n the theory of the analyss of two-way tables of nonaddtve da ta. Key words: Analyss of varance; covarance matrx; egenvalu es; nte racton; matrx ; resduals; two-way table; vacuum cleane r. 1. Introducton In a recent paper [1],1 a method has been presented for a parttonng of the row by column nteracton n two-way tables and the practcal us efulness of ths method as a tool n data analyss has bee n demonstrated. The method requred the calculaton, by Monte Carlo technques, of quanttes that are analogous to the "degrees of freedom" n ordnary analyss of varance. These new degrees of freedom were calculated as follows, Let (Z) be an m X n matrx of ndependent random normal devates (zero mean, unt varance). Consder the m X n matrx of resduals (d ), where d = Z + Z.. - Z. - Z., where a dot ndcates averagng over the ndex replaced by the dot. Then the new quanttes, denoted as degrees of freedom, are the expected values of th e egenvalues of the matrx product (d) (d) I (where I ndcates the transpose of a matrx). The expected values of these egenvalues are functons of m and n, and were tabulated as such. Professor John Tukey, of Prnceton Unversty, has ponted out that the degrees of freedom calculated by ths procedure are of far more general applcablty than ndcated n the paper n queston. The present paper provdes the theoretcal development from whch the more general results are obtaned. It s based on anew, mathematcally rgorous defnton for an nteracton matrx, from whch an mportant general result can be derved. 2. A Defnton of Interacton of Two-Way Tables Let (dj be an m X n matrx of normally dstrbuted varates, all of zero mean; and let the covarance of d and d,' be denoted by cov (d, dl' ), We wll say that (d) n an nteracton matrx ofr degrees offreedom by s degrees offreedom f cov(d, dll) s ofthe form COy (d, I dl' ) = [a '. b'] (T2, Fgures n brac kets nd cate the lt erature references at the end of ths paper. 149

2 where the mxm matrx AT= (a') s dempotent of rank r, and the nxn matrx Bs=(b') s dempotent of rank s. We wll say that AT and Bs are the covarance matrces assocated wth the nteracton matrx (d ). 3. The Egenvalues of (d)(d)' THEOREM I: If (d) s an m X n nteracton matrx wth assocated covarance matrces Ar and Bs, then the nonzero egenvalues of the matrx product (d)(d)' (where (d)' denotes the transpose of (d)) are the same as those of the matrx (t)(t)', where (t) s an r X s matrx of normally and ndependently dstrbuted varates t, of zero mean and common varance (Tt. PROOF: Let P = (PH') represent the m X m orthogonal matrx whose rows are the egenvectors of AT. Snce Ar s dempotent and of rank r, we have PA P' = (!d2) ' (2) where I,. s the dentty matrx or rank r. Smlarly, representng by Q = (q') the n X n orthogonal matrx whose rows are the egenvectors of Bs, we have From (2) and (3) we derve, respectvely, QBsQ' = (Wo) (3) and '" '"., {ow L.. L.. PkP~ akl = 0 k I for, ' ~ r for or ' > r for, ' ~ s for or ' > s, (4) (5) where Ok/s the Kronecker delta (Okl = 1 for k = l, and Okl = 0 for k '= /). Consder now the transformaton Then, or (t) = P(d)Q'. (t*) (t*)' = P(d)Q'Q(d)'P' (t*)(t*)' = P(d)(d)'P'. (6) (7) From (7) t follows that (t*) (t*)' and (d) (d)' have the same nonzero egenvalues. Furthermore, we have, on account of (6) m 11. ts = L L Pl.:qudku (8) Consequently, k II m n m n cov(t;', t(:') = L L L L PkqllP'lq'v cov (dku, d lv ). (9) k II I v Introducng (1) nto (9), we obtan m n m n cov (t;, t(:') = (T2 L L L L PkquP'lq'vaklblv k u I v 150

3 whch, as a result of eqs (4) and (5), becomes: o for for {,'~r,' ~ s or ' > r { or ' > s. (10) From (8) t also follows that the t'0 are normally dstrbuted wth E (t) = 0 for all and. Con sequently, because of (10), the tf) are equal to zero for all, ' > r or, ' > s. For, ' ~ rand, ' ~ s, t follows from (10) that the t'0 are normally and ndependently dstrbuted wth zero mean and varance (T2. We have already shown that (d) (d) 1 and (t* ) (t*) 1 have the same nonzero egenvalues. If we omt from the matrx (t*), all elements for whch > r, or > s (all of whch are zero), we obtan an r X s matrx, say (t), such that (t)() 1 has the same nonzero egenvalues as (t*)(t*) I, and consequently as (d) (d) I. QED. NOTE: We have assumed, n the defnton of an nteracton matrx, that the d are normally dstrbuted, and have proved that under ths assumpton, the t'0 defned by eq (8) (except for those that are dentcally zero), are ndependently and normally dstrbuted wth zero mean and varance (T2. We can generalze the defnton of an nteracton matrx by requrng only that the d be con dtonally normal, gven the a' and b" Snce the dstrbuton of the t does not depend on the au, and b l, t s uncondtonally normal wth zero mean and varance (T2, and Theorem I remans true under the more general defnton of an nteracton matrx. 4. Resduals from an Addtve Structure THEOREM II: Let (d u) be a matrx whose elements are { = 1 to m = 1 to n (1) where Yo are ndependent normal varates, such that E(yu) = /-L + P + Y, and whose common varance s (T2, and where {l, Ph Y are the usual estmates of the grand mean, the row effects and the column effects. Then (du) s an nteracton matrx of (m -1) degrees of freedom by (n - 1) degrees of freedom. PROOF: It s readly shown from (11) that cov (d, d 1 l ) = ( 8uI -,~)( 8' - ; )(T2. (12) The matrx A = ( 8U' - ~) s dempotent. Its trace s L ( 8, - ~) = m - 1. Hence ts rank s m -1. Smlarly, the matrx B = ( Ol - ;) s dempotent, of rank n - 1. Ths proves the theorem. 1 S. Resduals from the Vacuum Cleaner 2 LEMMA I: Let (Yu) be an m X n matrx of observatons Y, wth common varance (T2. Let (d,k) be the matrx of resduals obtaned from the Yu after extracton of the grand mean ({l), the row effects (P), the column effects (Y), and k sweeps of the vacuum cleaner [3]. Then the follown.g relaton holds:,.- du,k = Yu - ({l + P+Y) - L [K1Plql + CXlql +,BlPI] (13) 1= / 2 The result s co nlaned n Lemmas 1, II. and III are not novel. The y have been dc r ved I)re vo us ly, by a dffe re nt meth od, by W. L. Nchol son [ 2]. and arc prese nt ed here only for completeness. 151

4 n whch (2) (1) L P~ = L q = 1 for all!. (3) _ al- I, Pl-, ~La- l' ql= _---'=f3"'i-=i"", == ~Lm- I' Kl = L L d,i- IPlql f3l = L d,i- IPl - These relatons result from the defnton of the vacuum-cleaner process (see [3]), accordng to whch (14) and the quanttes K, p, q, a, {3, are calculated as above. LEMMA II: Usng the notaton of Lemma I, we have K1ql' and ~ PlPI' = all' J (15) for all 1, l' L qlql' = all' (16) PROOF: Let (t) = (tt, t21..., t,,) be any vector of n elements, such that L t = 0, and assume that the followng relatons hold for some gven value of l. (1) L tql+ l, = (2) " L.J t d J 1), I. = for all, where d,l satsfes eq.(14) of Lemma I. Then the followng relatons hold: for all. 152

5 PROOF: ( a ) L.. '" tql+2, = L.. '" t (3l + 1, = ---:= 1 == ~L/3 [+ I, ~L/3 [+ I, ) L t [L d,ipl+ 1, - KI + 1ql+ 1,] ) 1 - Ctl+1, L tql+ 1, - Pl+ 1, L t/31 + 1, = 0, because of (1), (2), and (a). Thus, f condtons (1) and (2) are fulflled for any value of l, they are also fulflled for any value l' > l. In that case, we have L tql'=o for' ;:?! l + 1. If we now make t == ql,, we can verfy that condtons (1) and (2) are fulflled. Hence, for l ' =l= l, and by defnton L qlql = 1. The relaton L PUPI' = Oll' s proved n a smlar way. LEMMA III: D ~fnng (d, k) as n LemmaI, we have cov (d,k, d'',k) = (1"2 ( Ow -! - p,pi') (O' - ~ - ±qlql')' l= 1 l = 1 PROOF: The lemma s easly verfed for k = 1. It s also readly shown, usng Lemma II, that f the lemma holds for k, t also holds for k + 1. Thus, the lemma s proved by mathematcal nducton on k. THEOREM III: Defne (d,k) as n Lemma I, 'lnd assume furthermore that the Y are normally and ndependently dstrbuted, wth E(Y) = J.t + P + Yh and common varance (1" 2 ; then the m X n matrx (d, k) s an nteracton matrx of (m - k - 1) by (n - k - 1) degree of freedom. PROOF: The theorem results at once from Lemm:a III amd the fact that a matrx of the form wth ~PlPl' = Oll', s dempotent and of rank m - k -1. (The trace of ths matrx s 3 The dstrbuton of th e d,a' s onl y condtof/all)' normal, gven all PI; and q l) va lu es. but as shown on p. 151, th s does not nvaldate Theore m I. Consequently, Theorem III :s true. 153

6 THEOREM IV: Let (Yu) be an m X n matrx of normal, ndependent varates Yu, whose means are E(y) = p.= P+ 'Y and whose common varance s (T2. Extract from Yu the mean fl, the row effect P> and the column effect Y, as well as k sweeps of the vacuum cleaner, and denote by (d,k) the matrx of resduals resultng from ths treatment. Then the nonzero egenvalues of the matrx product (d,k) (d,,, )' are the same as those of (t)(t)' where (t) s an (m-k-1) by (n-k-1) matrx (t) of normal, ndependent varatest of zero mean and common varance (T2. The theorem holds for all values of k, from k= 0 to k= mn [(m -1), (n -1)]. PROOF: The theorem s an mmedate consequence of Theorems I and III. 6. Prncpal Result and Summary As ndcated n the ntroducton, the present work was motvated by a desre to generalze results obtaned n connecton wth a new method of data analyss. Weare now n a poston to state ths generalzaton n the followng precse form. THEOREM V: Let (d) be an m X n nteracton matrx of r degrees of freedom by s degrees of freedom. Then the probablty dstrbuton of the egenvalues of the matrx product (d)(d)' depends only on rand s, and not on m and n. PROOF: The theorem follows at once from Theorem I. Our prncpal result s the followng: To study the probablty dstrbuton of the egenvalues of (d) (d); where (d) s any nteracton matrx wth r by s degrees of freedom, t s suffcent to study the probablty dstrbuton of the egenvalues of (t) (t) " where (t) s an r by s matrx whose elements are ndependent random normal devates (mean zero, unt varance). The probablty dstrbuton n queston s completely determned by the values of rand s. The author s greatly ndebted to Professor John Tukey for pontng out to hm the generalty of hs prevous results and thereby provdng the ncentve for the present work. 7. References [1] Mandel, John, The parttonng of nteracton n analyss of varance, Nat. Bur. Stand. (U.S.), 73B (Math. Sc.) No.4, (Oct. - Dec. 1969). [2] Ncholson, W. L., Notes for a Seres of Semnars on Tukey's Vacuum Cleaner (unpublshed), Battelle Memoral Insttute, Pacfc Nothwest Laboratory (Aprl 1, 1967).. [3] Tukey, J. W., The future of data analyss, Annals of Mathematcal Statstcs 33, No. 1, 1-67 (1962). (Paper 74B3-326) 154