R mp nq. (13.1) a m1 B a mn B

Transcription

1 This electronic version is for personal use and may not be duplicated or distributed Chapter 13 Kronecker Products 131 Definition and Examples Definition 131 Let A R m n, B R p q Then the Kronecker product (or tensor product) of A and B is defined as the matrix a 11 B a 1n B A B = R mp nq (131) a m1 B a mn B Obviously, the same definition holds if A and B are complex-valued matrices We restrict our attention in this chapter primarily to real-valued matrices, pointing out the extension to the complex case only where it is not obvious Example Let A = [ [ and B = Then A B = Note that B A = A B [ B 2B 3B 3B 2B B = For any B R p q, I 2 B = [ B B Replacing I 2 by I n yields a block diagonal matrix with n copies of B along the diagonal 3 Let B be an arbitrary 2 2 matrix Then B I 2 = b 11 b 12 b 11 b 12 b 21 b 22 b 21 b ajlbook 24/11/9 13:36 page 139 #147 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

2 This electronic version is for personal use and may not be duplicated or distributed 14 Chapter 13 Kronecker Products The extension to arbitrary B and I n is obvious 4 Let x R m, y R n Then x y = [ x 1 y T,,x m y T T = [x 1 y 1,,x 1 y n,x 2 y 1,,x m y n T R mn 5 Let x R m, y R n Then x y T = [x 1 y,,x m y T x 1 y 1 x 1 y n = x m y 1 x m y n = xy T R m n 132 Properties of the Kronecker Product Theorem 133 Let A R m n, B R r s, C R n p, and D R s t Then Proof: Simply verify that (A B)(C D) = AC BD ( R mr pt ) (132) a 11 B a 1n B c 11 D c 1p D (A B)(C D) = a m1 B a mn B c n1 D c np D n k=1 a 1kc k1 BD n k=1 a 1kc kp BD = n k=1 a mkc k1 BD n k=1 a mkc kp BD = AC BD Theorem 134 For all A and B, (A B) T = A T B T Proof: For the proof, simply verify using the definitions of transpose and Kronecker product Corollary 135 If A R n n and B R m m are symmetric, then A B is symmetric Theorem 136 If A and B are nonsingular, (A B) 1 = A 1 B 1 Proof: Using Theorem 133, simply note that (A B)(A 1 B 1 ) = I I = I ajlbook 24/11/9 13:36 page 14 #148 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

3 This electronic version is for personal use and may not be duplicated or distributed 132 Properties of the Kronecker Product 141 Theorem 137 If A R n n and B R m m are normal, then A B is normal Proof: (A B) T (A B) = (A T B T )(A B) by Theorem 134 = A T A B T B by Theorem 133 = AA T BB T since A and B are normal = (A B)(A B) T by Theorem 133 Corollary 138 If A R n n is orthogonal and B R m m is orthogonal, then A B is orthogonal Example 139 Let A = [ cos θ sin θ [ sin θ cos θ and B = cos φ sin φ sin φ cos φ Then it is easily seen that A is orthogonal with eigenvalues e ±jθ and B is orthogonal with eigenvalues e ±jφ The 4 4 matrix A B is then also orthogonal with eigenvalues e ±j(θ+φ) and e ±j(θ φ) Theorem 131 Let A R m n have a singular value decomposition U A A VA T B R p q have a singular value decomposition U B B VB T Then (U A U B )( A B )(V T A V T B ) and let yields a singular value decomposition of A B(after a simple reordering of the diagonal elements of A B and the corresponding right and left singular vectors) Corollary 1311 Let A R m n r have singular values σ 1 σ r > and let B R p q s have singular values τ 1 τ s > Then A B(or B A) has rs singular values σ 1 τ 1 σ r τ s > and rank(a B) = (ranka)(rankb) = rank(b A) Theorem 1312 Let A R n n have eigenvalues λ i,i n, and let B R m m have eigenvalues µ j,j m Then the mn eigenvalues of A B are λ 1 µ 1,,λ 1 µ m,λ 2 µ 1,,λ 2 µ m,,λ n µ m Moreover, if x 1,,x p are linearly independent right eigenvectors of A corresponding to λ 1,,λ p (p n), and z 1,,z q are linearly independent right eigenvectors of B corresponding to µ 1,,µ q (q m), then x i z j R mn are linearly independent right eigenvectors of A B corresponding to λ i µ j, i p, j q Proof: The basic idea of the proof is as follows: (A B)(x z) = Ax Bz = λx µz = λµ(x z) If A and B are diagonalizable in Theorem 1312, we can take p = n and q = m and thus get the complete eigenstructure of A B In general, if A and B have Jordan form ajlbook 24/11/9 13:36 page 141 #149 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

4 This electronic version is for personal use and may not be duplicated or distributed 142 Chapter 13 Kronecker Products decompositions given by P 1 AP = J A and Q 1 BQ = J B, respectively, then we get the following Jordan-like structure: (P Q) 1 (A B)(P Q) = (P 1 Q 1 )(A B)(P Q) = (P 1 AP ) (Q 1 BQ) = J A J B Note that J A J B, while upper triangular, is generally not quite in Jordan form and needs further reduction (to an ultimate Jordan form that also depends on whether or not certain eigenvalues are zero or nonzero) A Schur form for A B can be derived similarly For example, suppose P and Q are unitary matrices that reduce A and B, respectively, to Schur (triangular) form, ie, P H AP = T A and Q H BQ = T B (and similarly if P and Q are orthogonal similarities reducing A and B to real Schur form) Then (P Q) H (A B)(P Q) = (P H Q H )(A B)(P Q) = (P H AP ) (Q H BQ) = T A T B Corollary 1313 Let A R n n and B R m m Then 1 Tr(A B) = (TrA)(TrB) = Tr(B A) 2 det(a B) = (det A) m (det B) n = det(b A) Definition 1314 Let A R n n and B R m m Then the Kronecker sum (or tensor sum) of A and B, denoted A B, is the mn mn matrix (I m A) + (B I n ) Note that, in general, A B = B A Example Let A = and B = [ Then A B = (I 2 A)+(B I 3 ) = The reader is invited to compute B A = (I 3 B)+(A I 2 ) and note the difference with A B ajlbook 24/11/9 13:36 page 142 #15 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

5 This electronic version is for personal use and may not be duplicated or distributed 132 Properties of the Kronecker Product Recall the real JCF M I M I J = M R 2k 2k, I M I M [ α β where M = Define β α 1 1 E k = R k k 1 Then J can be written in the very compact form J = (I k M)+(E k I 2 ) = M E k Theorem 1316 Let A R n n have eigenvalues λ i,i n, and let B R m m have eigenvalues µ j,j m Then the Kronecker sum A B = (I m A) + (B I n ) has mn eigenvalues λ 1 + µ 1,,λ 1 + µ m,λ 2 + µ 1,,λ 2 + µ m,,λ n + µ m Moreover, if x 1,,x p are linearly independent right eigenvectors of A corresponding to λ 1,,λ p (p n), and z 1,,z q are linearly independent right eigenvectors of B corresponding to µ 1,,µ q (q m), then z j x i R mn are linearly independent right eigenvectors of A B corresponding to λ i + µ j, i p, j q Proof: The basic idea of the proof is as follows: [(I m A) + (B I n )(z x) = (z Ax) + (Bz x) = (z λx) + (µz x) = (λ + µ)(z x) If A and B are diagonalizable in Theorem 1316, we can take p = n and q = m and thus get the complete eigenstructure of A B In general, if A and B have Jordan form decompositions given by P 1 AP = J A and Q 1 BQ = J B, respectively, then [(Q I n )(I m P) 1 [(I m A) + (B I n )[(Q I n )(I m P) =[(I m P) 1 (Q I n ) 1 [(I m A) + (B I n )[(Q I n )(I m P) =[(I m P 1 )(Q 1 I n )[(I m A) + (B I n )[(Q I n )(I m P) = (I m J A ) + (J B I n ) is a Jordan-like structure for A B ajlbook 24/11/9 13:36 page 143 #151 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

6 This electronic version is for personal use and may not be duplicated or distributed 144 Chapter 13 Kronecker Products A Schur form for A B can be derived similarly Again, suppose P and Q are unitary matrices that reduce A and B, respectively, to Schur (triangular) form, ie, P H AP = T A and Q H BQ = T B (and similarly if P and Q are orthogonal similarities reducing A and B to real Schur form) Then [(Q I n )(I m P) H [(I m A) + (B I n )[(Q I n )(I m P)=(I m T A ) + (T B I n ), where [(Q I n )(I m P)=(Q P)is unitary by Theorem 133 and Corollary Application to Sylvester and Lyapunov Equations In this section we study the linear matrix equation AX + XB = C, (133) where A R n n, B R m m, and C R n m This equation is now often called a Sylvester equation in honor of JJ Sylvester who studied general linear matrix equations of the form k A i XB i = C i=1 A special case of (133) is the symmetric equation AX + XA T = C (134) obtained by taking B = A T When C is symmetric, the solution X R n n is easily shown also to be symmetric and (134) is known as a Lyapunov equation Lyapunov equations arise naturally in stability theory The first important question to ask regarding (133) is, When does a solution exist? By writing the matrices in (133) in terms of their columns, it is easily seen by equating the ith columns that m Ax i + Xb i = c i = Ax i + b ji x j These equations can then be rewritten as the mn mn linear system A + b 11 I b 21 I b m1 I b 12 I A+ b 22 I b m2 I x 1 = x b 1m I b 2m I A + b mm I m j=1 c 1 c m (135) The coefficient matrix in (135) clearly can be written as the Kronecker sum (I m A) + (B T I n ) The following definition is very helpful in completing the writing of (135) as an ordinary linear system ajlbook 24/11/9 13:36 page 144 #152 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

7 This electronic version is for personal use and may not be duplicated or distributed 133 Application to Sylvester and Lyapunov Equations 145 Definition 1317 Let c i R n denote the columns of C R n m so that C = [c 1,,c m Then vec(c) is defined to be the mn-vector formed by stacking the columns of C on top of c 1 one another, ie, vec(c) = c m R mn Using Definition 1317, the linear system (135) can be rewritten in the form [(I m A) + (B T I n )vec(x) = vec(c) (136) There exists a unique solution to (136) if and only if [(I m A) + (B T I n ) is nonsingular But [(I m A) + (B T I n ) is nonsingular if and only if it has no zero eigenvalues From Theorem 1316, the eigenvalues of [(I m A) + (B T I n ) are λ i + µ j, where λ i (A), i n, and µ j (B), j m We thus have the following theorem Theorem 1318 Let A R n n, B R m m, and C R n m Then the Sylvester equation AX + XB = C (137) has a unique solution if and only if A and B have no eigenvalues in common Sylvester equations of the form (133) (or symmetric Lyapunov equations of the form (134)) are generally not solved using the mn mn vec formulation (136) The most commonly preferred numerical algorithm is described in [2 First A and B are reduced to (real) Schur form An equivalent linear system is then solved in which the triangular form of the reduced A and B can be exploited to solve successively for the columns of a suitably transformed solution matrix X Assuming that, say, n m, this algorithm takes only O(n 3 ) operations rather than the O(n 6 ) that would be required by solving (136) directly with Gaussian elimination A further enhancement to this algorithm is available in [6 whereby the larger of A or B is initially reduced only to upper Hessenberg rather than triangular Schur form The next few theorems are classical They culminate in Theorem 1324, one of many elegant connections between matrix theory and stability theory for differential equations Theorem 1319 Let A R n n, B R m m, and C R n m Suppose further that A and B are asymptotically stable (a matrix is asymptotically stable if all its eigenvalues have real parts in the open left half-plane) Then the (unique) solution of the Sylvester equation AX + XB = C (138) can be written as X = + e ta Ce tb dt (139) Proof: Since A and B are stable, λ i (A) + λ j (B) = for all i, j so there exists a unique solution to (138) by Theorem 1318 Now integrate the differential equation Ẋ = AX+XB (with X() = C) on[, + ): + ( + ) lim X(t) X() = A X(t)dt + X(t)dt B (131) t + ajlbook 24/11/9 13:36 page 145 #153 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

8 This electronic version is for personal use and may not be duplicated or distributed 146 Chapter 13 Kronecker Products Using the results of Section 1116, it can be shown easily that lim t + eta = lim t + etb = Hence, using the solution X(t) = e ta Ce tb from Theorem 116, we have that lim X(t) = t + Substituting in (131) we have ( + ) ( + ) C = A e ta Ce tb dt + e ta Ce tb dt B and so X = + e ta Ce tb dt satisfies (138) Remark 132 An equivalent condition for the existence of a unique solution to AX + XB = C is that [ A C [ B be similar to A [ B (via the similarity I X I ) Theorem 1321 Let A, C R n n Then the Lyapunov equation AX + XA T = C (1311) has a unique solution if and only if A and A T have no eigenvalues in common If C is symmetric and (1311) has a unique solution, then that solution is symmetric Remark 1322 If the matrix A R n n has eigenvalues λ 1,,λ n, then A T has eigenvalues λ 1,, λ n Thus, a sufficient condition that guarantees that A and A T have no common eigenvalues is that A be asymptotically stable Many useful results exist concerning the relationship between stability and Lyapunov equations Two basic results due to Lyapunov are the following, the first of which follows immediately from Theorem 1319 Theorem 1323 Let A, C R n n and suppose further that A is asymptotically stable Then the (unique) solution of the Lyapunov equation can be written as X = AX + XA T = C + e ta Ce tat dt (1312) Theorem 1324 A matrix A R n n is asymptotically stable if and only if there exists a positive definite solution to the Lyapunov equation where C = C T < AX + XA T = C, (1313) Proof: Suppose A is asymptotically stable By Theorems 1321 and 1323 a solution to (1313) exists and takes the form (1312) Now let v be an arbitrary nonzero vector in R n Then + v T Xv = (v T e ta )( C)(v T e ta ) T dt ajlbook 24/11/9 13:36 page 146 #154 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

9 This electronic version is for personal use and may not be duplicated or distributed 133 Application to Sylvester and Lyapunov Equations 147 Since C > and e ta is nonsingular for all t, the integrand above is positive Hence v T Xv > and thus X is positive definite Conversely, suppose X = X T > and let λ (A) with corresponding left eigenvector y Then >y H Cy = y H AXy + y H XA T y = (λ + λ)y H Xy Since y H Xy >, we must have λ + λ = 2Reλ< Since λ was arbitrary, A must be asymptotically stable Remark 1325 The Lyapunov equation AX + XA T = C can also be written using the vec notation in the equivalent form [(I A) + (A I)vec(X) = vec(c) A subtle point arises when dealing with the dual Lyapunov equation A T X + XA = C The equivalent vec form of this equation is [(I A T ) + (A T I)vec(X) = vec(c) However, the complex-valued equation A H X + XA = C is equivalent to [(I A H ) + (A T I)vec(X) = vec(c) The vec operator has many useful properties, most of which derive from one key result Theorem 1326 For any three matrices A, B, and C for which the matrix product ABC is defined, vec(abc) = (C T A)vec(B) Proof: The proof follows in a fairly straightforward fashion either directly from the definitions or from the fact that vec(xy T ) = y x An immediate application is to the derivation of existence and uniqueness conditions for the solution of the simple Sylvester-like equation introduced in Theorem 611 Theorem 1327 Let A R m n, B R p q, and C R m q Then the equation AXB = C (1314) has a solution X R n p if and only if AA + CB + B = C, in which case the general solution is of the form X = A + CB + + Y A + AY BB +, (1315) where Y R n p is arbitrary The solution of (1314) is unique if BB + A + A = I Proof: Write (1314) as (B T A)vec(X) = vec(c) (1316) ajlbook 24/11/9 13:36 page 147 #155 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

10 This electronic version is for personal use and may not be duplicated or distributed 148 Chapter 13 Kronecker Products by Theorem 1326 This vector equation has a solution if and only if (B T A)(B T A) + vec(c) = vec(c) It is a straightforward exercise to show that (M N) + = M + N + Thus, (1316) has a solution if and only if vec(c) = (B T A)((B + ) T A + )vec(c) =[(B + B) T AA + vec(c) = vec(aa + CB + B) and hence if and only if AA + CB + B = C The general solution of (1316) is then given by vec(x) = (B T A) + vec(c) +[I (B T A) + (B T A)vec(Y ), where Y is arbitrary This equation can then be rewritten in the form vec(x) = ((B + ) T A + )vec(c) +[I (BB + ) T A + Avec(Y ) or, using Theorem 1326, X = A + CB + + Y A + AY BB + The solution is clearly unique if BB + A + A = I EXERCISES 1 For any two matrices A and B for which the indicated matrix product is defined, show that (vec(a)) T (vec(b)) = Tr(A T B) In particular, if B R n n, then Tr(B) = vec(i n ) T vec(b) 2 Prove that for all matrices A and B, (A B) + = A + B + 3 Show that the equation AXB = C has a solution for all C if A has full row rank and B has full column rank Also, show that a solution, if it exists, is unique if A has full column rank and B has full row rank What is the solution in this case? 4 Show that the general linear equation k A i XB i = C can be written in the form i=1 [B T 1 A 1 + +B T k A kvec(x) = vec(c) ajlbook 24/11/9 13:36 page 148 #156 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

11 This electronic version is for personal use and may not be duplicated or distributed Exercises Let x R m and y R n Show that x T y = yx T 6 Let A R n n and B R m m (a) Show that A B 2 = A 2 B 2 (b) What is A B F in terms of the Frobenius norms of A and B? Justify your answer carefully (c) What is the spectral radius of A B in terms of the spectral radii of A and B? Justify your answer carefully 7 Let A, B R n n (a) Show that (I A) k = I A k and (B I) k = B k I for all integers k (b) Show that e I A = I e A and e B I = e B I (c) Show that the matrices I A and B I commute (d) Show that e A B = e (I A)+(B I) = e B e A (Note: This result would look a little nicer had we defined our Kronecker sum the other way around However, Definition 1314 is conventional in the literature) 8 Consider the Lyapunov matrix equation (1311) with [ 1 A = 1 and C the symmetric matrix [ 2 2 Clearly [ 1 X s = 1 is a symmetric solution of the equation Verify that [ X ns = is also a solution and is nonsymmetric Explain in light of Theorem Block Triangularization: Let [ A B S = C D where A R n n and D R m m It is desired to find a similarity transformation of the form [ I T = X I such that T 1 ST is block upper triangular, ajlbook 24/11/9 13:36 page 149 #157 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html

12 This electronic version is for personal use and may not be duplicated or distributed 15 Chapter 13 Kronecker Products (a) Show that S is similar to [ A + BX B D XB if X satisfies the so-called matrix Riccati equation C XA + DX XBX = (b) Formulate a similar result for block lower triangularization of S 1 Block Diagonalization: Let S = [ A B D where A R n n and D R m m It is desired to find a similarity transformation of the form [ I Y T = I such that T 1 ST is block diagonal (a) Show that S is similar to [ A D if Y satisfies the Sylvester equation, AY YD = B (b) Formulate a similar result for block diagonalization of [ A S = C D ajlbook 24/11/9 13:36 page 15 #158 From "Matrix Analysis for Scientists and Engineers" Alan J Laub Buy this book from SIAM at wwwec-securehostcom/siam/ot91html