An Index Notation for Tensor Products

Transcription

1 APPENDIX 6 An Index Noaion for Tensor Producs 1 Bases for Vecor Spaces Consider an ideniy marix of order N, which can be wrien as follows: e (1) [ e 1 e 2 e N ] = = e e N On he LHS, he marix is expressed as a collecion of column vecors, denoed by e i ; i = 1, 2,, N, which form he basis of an ordinary N-dimensional Euclidean space, which is he primal space On he RHS, he marix is expressed as a collecion of row vecors e ; = 1, 2,, N, which form he basis of he conugae dual space The basis vecors can be used in specifying arbirary vecors in boh spaces In he primal space, here is he column vecor (2) a = X i a i e i = (a i e i ), and in he dual space, here is he row vecor (3) b 0 = X b e = (b e ) Here, on he RHS, here is a noaion ha replaces he summaion signs by parenheses When a basis vecor is enclosed by pahenheses, summaions are o be aken in respec of he index or indices ha i carries Usually, such an index will be associaed wih a scalar elemen ha will also be found wihin he parenheses The advanage of his noaion will become apparen a a laer sage, when he summaions are over several indices A vecor in he primary space can be convered o a vecor in he conugae dual space and vice versa by he operaion of ransposiion Thus a 0 = (a i e i ) is formed via he conversion e i e i whereas b = (b e ) is formed via he conversion e e 1

2 DSG POLLOCK : ECONOMETRICS 2 Elemenary Tensor Producs A ensor produc of wo vecors is an ouer produc ha enails he pairwise producs of he elemens of boh vecor Consider wo primal vecors (4) a = [a ; = 1, T ] = [a 1, a 2,, b T ] 0 b = [b ; = 1,, M] = [b 1, b 2,, b M ] 0, which need no be of he same order Then, wo kinds of ensor producs can be defined Firs, here are covarian ensor producs The covarian produc of a and b is a column vecor in a primal space: (5) a b = X X a b (e e ) = (a b e ) Here, he elemens are arrayed in a long column in an order ha is deermined by he lexicographic variaion of he indices and Thus, he index undergoes a complee cycles from = 1 o = M wih each incremen of he index in he manner ha is familiar from dicionary classificaions Thus (6) a b = a 1 b a 2 b a T b and = [a 1b 1,, a 1 b M, a 2 b 1,, a 2 b M,, a T b 1,, a T b M ] 0 A covarian ensor produc can also be formed from he row vecors a 0 and b 0 of he dual space Thus, here is (7) a 0 b 0 = X X a b (e e ) = (a b e ) I will be observed ha his is us he ranspose of a b Tha is o say (8) (a b) 0 = a 0 b 0 or, equivalenly (a b e ) 0 = (a b e ) The order of he vecors in a covarian ensor produc is crucial, since, as once can easily verify, i is he case ha (9) a b 6= b a and a 0 b 0 6= b 0 a 0 The second kind of ensor produc of he wo vecors is a so-called conravarian ensor produc: (10) a b 0 = b 0 a = X X a b (e e ) = (a b e ) (11) This is us he familiar marix produc ab 0, which can be wrien variously as a 1 b a 1 b 1 a 1 b 2 a 1 b M a 2 b a 2 b 1 a 2 b 2 a 2 b M a T b = [ b 1a b 2 a b M a ] = 2 a T b 1 a T b 2 a T b M

3 1: MATRIX ALGEBRA Observe ha (12) (a b 0 ) 0 = a 0 b or, equivalenly, (a b e ) 0 = (a b e ) We now propose o dispense wih he summaion signs and o wrie he various vecors as follows: (13) a = (a e ), a 0 = (a e ) and b = (a e ), b 0 = (b e ) As before, he convenion here, is ha, when he producs are surrounded by parenheses, summaions are o be ake in respec of he indices ha are associaed wih he basis vecors The convenion can be applied o provide summary represenaions of he producs under (5), (7) and (10): a b 0 = (a e ) (b e ) = (a b e ), (14) a 0 b 0 = (a e ) (b e ) = (a b e ), (15) a b = (a e ) (b e ) = (a b e ) (16) Such producs are described as decomposable ensors 3 Non-decomposable Tensor Producs Non-decomposable ensors are he resul of aking weighed sums of decomposable ensors Consider an arbirary marix X = [x ] of order T M This can be expressed as he following weighed sum of he conravarian ensor producs formed from he basis vecors: (17) X = (x e ) = X X x (e e ) The indecomposabiliy lies in he fac ha he elemens x canno be wrien as he producs of an elemen indexed by and an elemen indexed by From X = (x e ), he following associaed ensors producs may be derived: X 0 = (x e ), (18) X r = (x e ), (19) X c = (x e ) (20) Here, X 0 is he ransposed marix, whereas X c is a long column vecor and X r is a long row vecor Noice ha, in forming X c and X r from X, he index ha moves assumes a posiion a he head of he sring of indices o which i is oined I is eviden ha (21) X r = X 0c0 and X c = X 0r0 3

4 DSG POLLOCK : ECONOMETRICS Thus, i can be seen ha X c and X r are no relaed o each oher by simple ransposiions A consequence of his is ha he indices of he elemens in X c follow he reverse of a lexicographic ordering Example Consider he equaion (22) y = µ + γ + δ + ε wherein = 1,, T and = 1,, M This relaes o a wo-way analysis of variance For a concree inerpreaion, we may imagine ha y is an observaion aken a ime in he h region Then, he parameer γ represens an effec ha is common o all observaions aken a ime, whereas he parameer δ represens a characerisic of he h region ha prevails hrough ime In ordinary marix noaion, he se of T M equaions becomes (23) Y = µι T ι 0 M + γι 0 M + ι T δ 0 + E, where Y = [y ] and E = [ε ] are marices of order T M, γ = [γ 1,, γ T ] 0 and δ = [δ 1,, δ M ] 0 are vecors of orders T and M respecively, and ι T and ι M are vecors of unis whose orders are indicaed by heir subscrips In erms of he index noaion, he T M equaions are represened by (24) (y e ) = µ(e ) + (γ e ) + (δ e ) + (ε e ) An illusraion is provided by he case where T = M = 3 Then equaions (23) and (24) represen he following srucure: (25) y 11 y 12 y 13 y 21 y 22 y 23 = µ γ 1 γ 1 γ 1 γ 2 γ 2 γ 2 y 31 y 32 y γ 3 γ 3 γ 3 + δ 1 δ 2 δ 3 δ 1 δ 2 δ 3 + ε 11 ε 12 ε 13 ε 21 ε 22 ε 23 δ 1 δ 2 δ 3 ε 31 ε 32 ε 33 4 Muliple Tensor Producs The ensor produc enails an associaive operaion ha combines marices or vecors of any order Le B = [b l ] and A = [a ki ] be arbirary marices of orders n and s m respecively Then, heir ensor produc B A, which is also know as a Kronecker produc, is defined in erms of he index noaion by wriing (26) (b l e l ) (a kie i k) = (b l a ki e i lk ) Here, e i lk sands for a marix of order s mn wih a uni in he row indexed by lk he {(l 1)s + k}h row and in he column indexed by i he {( 1)m + i}h column and wih zeros elsewhere 4

5 1: MATRIX ALGEBRA In he marix array, he row indices lk follow a lexicocographic order, as do he column indices i Also, he indices lk are no ordered relaive o he indices i Tha is o say, e i lk = e l e k e e i (27) = e e i e l e k = e e l e k e i = e l e e i e k = e l e e k e i = e e l e i e k The virue of he index noaion is ha i makes no disincion amongs hese various producs on he RHS unless a disincion can be found beween such expressions as e i l k and e i l k For an example, consider he Kronecker of wo marices as follows: (28) b11 b 12 a11 a 12 = b 21 b 22 = b 11 b 21 b 12 b 22 b 11 a 11 b 11 a 12 b 12 a 11 b 12 a 12 b 11 a 21 b 11 a 22 b 12 a 21 b 12 a 22 b 21 a 11 b 21 a 12 b 22 a 11 b 22 a 12 b 21 a 21 b 21 a 22 b 22 a 21 b 22 a 22 Here, i can be see ha he composie row indices lk, associaed wih he elemens b l a ki, follow he lexicographic sequence {11, 12, 21, 22} The column indices follow he same sequence 5 Composiions In order o demonsrae he rules of marix composiion, le us consider he marix equaion (29) Y = AXB 0, which can be consrued as a mapping from X o Y In he index noaion, his is wrien as (30) (y kl e l k) = (a ki e i k)(x i e i )(b le l ) = ({a ki x i b l }e l k) Here, here is (31) {a ki x i b l } = X i X a ki x i b l ; 5

6 DSG POLLOCK : ECONOMETRICS which is o say ha he braces surrounding he expression on he LHS are o indicae ha summaions are aken wih respec o he repeaed indices i and, which are associaed wih he basis vecors The operaion of composing wo facors depends upon he cancellaion of a superscrip (column) index, or sring of indices, in he leading facor wih an equivalen subscrip (row) index, or sring of indices, in he following facor The marix equaion of (29) can be vecorised in a variey of ways In order o represen he mapping from X c = (x i e i ) o Y c = (y kl e lk ), we may wrie (32) (y kl e lk ) = ({a ki x i b l }e lk ) = (a ki b l e i lk )(x ie i ) Noice ha he produc a ki b l wihin (a ki b l e i lk ) does no need o be surrounded by braces since i conains no repeaed indices Neverheless, here would be no harm in wriing {a ki b l } The marix (a ki b l e i lk ) is decomposable Tha is o say (33) (a ki b l e i lk ) = (b le l ) (a kie i k) = B A; and, herefore, he vecorised form of equaion (29) is (34) Y c = (AXB 0 ) c = (B A)X c Example The equaion under (22), which relaes o a wo-way analysis of variance, can be vecorised o give (35) (y e ) = µ(e ) + (e )(γ e ) + (e )(δ e ) + (ε e ) Using he noaion of he Kronecker produc, his can also be rendered as (36) Y c = µ(ι M ι T ) + (ι M I T )γ + (I M ι T )δ + E c = Xβ + E c The laer can also be obained by applying he rule of (34) o equaion (23) The various elemens of (23) have been vecorised as follows: (µι T ι 0 M) c = (ι T µι 0 M) c = (ι M ι T )µ, (37) (γι 0 M) c = (I γι 0 M) c = (ι M I T )γ, (ι T δ 0 ) c = (ι T δ 0 I M ) c = (I M ι T )δ 0c, δ 0c = δ Also, here is (ι M ι T )µ = µ(ι M ι T ), since µ is a scalar elemen ha can be ransposed or freely associaed wih any facor of he expression 6

7 1: MATRIX ALGEBRA In comparing (35) and (36), we see, for example, ha (e ) = (e ) (e ) = ι M I T We recognise ha (e ) is he sum over he index of he marices of order T which have a uni in he h diagonal posiion and zeros elsewhere; and his sum amouns, of course, o he ideniy marix of order T The vecorised form of equaion (25) is (38) y 11 y 21 y 31 y 12 y 22 y 32 y 13 y 23 y 33 = µ γ 1 γ 2 γ 3 + δ 1 δ 2 δ 3 ε 11 ε 21 ε 31 ε 12 ε 22 ε 32 ε 13 ε 23 ε 33 6 Rules for Decomposable Tensor Producs The following rules govern he decomposable ensors produc of marices, which are commonly described as Kronecker producs: (39) (i) (A B)(C D) = AC BD, (ii) (A B) 0 = A 0 B 0, (iii) A (B + C) = (A B) + (A C), (iv) λ(a B) = λa B = A λb, (v) (A B) 1 = (A 1 B 1 ) The Kronecker produc is non-commuaive, which is o say ha A B 6= B A However, observe ha (40) A B = (A I)(I B) = (I B)(A I) 7